Learning a Hierarchy of Discriminative Space-Time Neighborhood Features for Human Action Recognition

Recent work shows how to use local spatio-temporal features to learn models of realistic human actions from video. However, existing methods typically rely on a predefined spatial binning of the local descriptors to impose spatial information beyond a pure "bag-of-words" model, and thus may fail to capture the most informative space-time relationships. We propose to learn the shapes of space-time feature neighborhoods that are most discriminative for a given action category. Given a set of training videos, our method first extracts local motion and appearance features, quantizes them to a visual vocabulary, and then forms candidate neighborhoods consisting of the words associated with nearby points and their orientation with respect to the central interest point. Rather than dictate a particular scaling of the spatial and temporal dimensions to determine which points are near, we show how to learn the class-specific distance functions that form the most informative configura...

• Type: Conference
• Year: 2010
• Where: CVPR
• Authors: Adriana Kovashka, Kristen Grauman
• Keywords: Computer Vision, CVPR 2010, Informative Space-time Relationships, Local Spatio-temporal Features, Space-time Feature Neighborhoods