We present a motion classification approach to detect movements of interest (abnormal motion) based on local feature modeling within spatio-temporal detectors. The modeling is performed using motion vectors and local detectors. The detectors are trained independently for learning abnormal motion based on labeled samples. Each detector is assigned an abnormality score, both in space and time, which is the basis of the final classification. The spatial relationship across detectors is used to discriminate simultaneous occurrences of abnormal motion. The performance of the proposed method is evaluated on 52 hours of the multi-camera surveillance dataset of the TRECVID [1] 2010 challenge.