A correct video segmentation, namely the detection of moving objects within a scene plays a very important role in many application in safety, surveillance, trafic monitoring and object detection. The main objective of this paper is to implement an effective background segmentation algorithm for corner sets extracted from video sequences. A dynamic prototype of the structure of background corners is produced and incoming corners are classified using a Fuzzy ARTMAP Neural Network and labeled as pertaining to the background or foreground using a spatial clustering method. Finally the accuracy of the proposed algorithm is evaluated using PETS2006 benchmark data.