Since the H.264 video coding standard becomes quite popular for transmission of diverse multimedia data due to its high coding efficiency, bit allocation and buffer management schemes are required for various applications. In this paper, we propose a new bit rate control algorithm for H.264 based on an accurate bit allocation strategy, where we introduce a reference quantization parameter (QP) table for the intra frame. For the inter frame, we propose a new bit allocation scheme by adjusting QP considering overhead bits and ratedependent mode selection. Experimental results demonstrate that the number of the actual coding bits and the number of estimated bits match well such that we can avoid severe degradation of picture quality at the end of the group of pictures (GOP). Moreover, we have improved coding efficiency and average PSNR by up to 0.25dB.