Most codecs such as MPEG-1,2 are based on processing pixels in predetermined tiles of square blocks. The processing of each block is independent of the object-based contents that may be present in these blocks. Segmentation of moving object blocks enables, for example, efficient distribution of bits among consecutive frames in video sequences by allocating more bits to moving object blocks at the expense of the background blocks. This paper proposes an algorithm which determines whether the MPEG blocks contain static or moving objects. The proposed algorithm is based on temporal statistical information obtained by multiple comparisons of consecutive frames. A dynamic process of gathering statistical information is converged into three clusters with different characterizations. Automatic classification determines which blocks belong to either a moving object (foreground) or a static object (background). The number of look-ahead frame comparisons, which is required to determine the type ...