—In this paper, we address the key problem in real-time video coding, the rate-distortion (R-D) tradeoff. As most video coding applications employ color images, we analyze the R-D modeling problem of the color image sequence. We investigate that the R-D characteristics of color video signal in traditional transform-based video coding systems should be modeled for the luminance and chrominance components separately. So we propose separable R-D models for color video coding. The feedback from the encoder buffer is analyzed by a control-theoretic adaptation approach to avoid buffer overflow and underflow. To achieve smooth video quality and satisfy the delay constraints in real-time applications, a novel R-D tradeoff controller is designed. In the proposed R-D tradeoff framework, both the quality variation and buffer safety are considered. Extensive simulation results demonstrate that the proposed approach can achieve smooth video quality without sacrificing overall quality while ef...