This paper presents a novel learning method for human action detection in video sequences. The detecting problem is not limited in controlled settings like stationary background or invariant illumination, but studied in real scenarios. Spatio-temporal volume analysis for actions is adopted to solve the problem. To develop effective representation while remaining resistant to background motions, only motion information is exploited to define suitable descriptors for action volumes. On the other hand, action models are learned by using boosting techniques to select discriminative features for efficient classification. This paper also shows how the proposed method enables learning efficient action detectors, and validates them on publicly available datasets. Keywords Action retrieving · Activity analysis · Video understanding · Visual surveillance · Local motion histograms