From Videos to Verbs: Mining Videos for Activities using a Cascade of Dynamical Systems

Clustering video sequences in order to infer and extract activities from a single video stream is an extremely important problem and has significant potential in video indexing, surveillance, activity discovery and event recognition. Clustering a video sequence into activities requires one to simultaneously recognize activity boundaries (activity consistent subsequences) and cluster these activity subsequences. In order to do this, we build a generative model for activities (in video) using a cascade of dynamical systems and show that this model is able to capture and represent a diverse class of activities. We then derive algorithms to learn the model parameters from a video stream and also show how a single video sequence may be clustered into different clusters where each cluster represents an activity. We also propose a novel technique to build affine, view, rate invariance of the activity into the distance metric for clustering. Experiments show that the clusters found by the alg...

• Type: Conference
• Year: 2007
• Where: CVPR
• Authors: Pavan K. Turaga, Ashok Veeraraghavan, Rama Chellappa
• Keywords: Activity Boundaries, Activity Consistent Subsequences, Activity Discovery, Computer Vision, CVPR 2007, Single Video Sequence, Single Video Stream