A motion segmentation algorithm based on factorization method and discriminant criterion is proposed. This method uses a feature with the most useful similarities for grouping, selected using motion information calculated by factorization method and discriminant criterion. A group is extracted based on discriminant analysis for the selected feature's similarities. The same procedure is applied recursively to the remaining features to extract other groups. This grouping is robust against noise and outliers because features with no useful information are automatically rejected. Numerical computation is simple and stable. No prior knowledge is needed on the number of objects. Experimental results are shown for synthetic data and real image sequences.