In this paper, we propose a new method to extract moving objects from a video stream without any motion estimation. The objective is to obtain a method robust to noise, large motions and ghost phenomena. Our approach consists in a frame differencing strategy combined with a hierarchical segmentation approach. First, we propose to extract moving edges with a new robust difference scheme, based on the spatial gradient. In the second stage, the moving regions are extracted from previously detected moving edges by using a hierarchical segmentation. The obtained moving objects description is represented as an adjacency graph. The method is validated on real sequences in the context of video-surveillance, assuming a static camera hypothesis.