Abstract. In this paper, we introduce the composition problem associated with object-based video coding and propose a solution to this problem. Although the object-based rate control algorithm can provide the overall coding gain, it may create the composition problem due to different temporal resolutions in object encoding. By checking shape changes of video objects, we can minimize the possibility of the composition problem at the encoder. At the decoder, we can also apply hole detection and recovery algorithms to eliminate the effect of the composition problem to the human visual system.