The recent JVT video coding scheme (MPEG-4 AVC/H.264) is a promising technique due to its high coding efficiency. Hypothetical Reference Decoder (HRD) is a very important part in JVT video coding, which represents a set of normative requirements on bitstream for the purpose of avoiding buffer overflow and underflow. The problem of HRD requirements can be solved by rate control. This paper proposes an effective rate control scheme for JVT video coding with HRD considerations. First, bit allocation with HRD constraints is presented, and second, based on a simple rate distortion model, a single pass rate control is implemented on both frame level and macroblock level. Experimental results show that the proposed rate control algorithm can achieve the target bit rate with very little bit rate or image quality fluctuation, and meanwhile it can well meet the HRD requirements. Furthermore, the proposed algorithm is so simple that it only introduces little computation complexity. Therefore, it...