This paper suggests a novel segmentation algorithm for separating moving objects from the background in video sequences without any prior information of the sequence nature. We formulate the problem as a connectivity analysis of region adjacency graph (RAG) based on temporal information. The nodes of the RAG represent homogeneous regions and the edges represent temporal information, which is obtained by frames comparison iterations. Connectivity analysis of the RAG nodes is performed after each frames comparison by a breadth first search (BFS) based algorithm. The set of nodes, which achieve maximum weight of theirs surrounding edges are considered as moving object. The number of comparisons that are needed for temporal information is automatically determined.