Parsing Video Events with Goal inference and Intent Prediction

In this paper, we present an event parsing algorithm based on Stochastic Context Sensitive Grammar (SCSG) for understanding events, inferring the goal of agents, and predicting their plausible intended actions. The SCSG represents the hierarchical compositions of events and the temporal relations between the sub-events. The alphabets of the SCSG are atomic actions which are defined by the poses of agents and their interactions with objects in the scene. The temporal relations are used to distinguish events with similar structures, interpolate missing portions of events, and are learned from the training data. In comparison with existing methods, our paper makes the following contributions. i) We define atomic actions by a set of relations based on the fluents of agents and their interactions with objects in the scene. ii) Our algorithm handles events insertion and multi-agent events, keeps all possible interpretations of the video to preserve the ambiguities, and achieves the globa...

• Type: Journal
• Year: 2011
• Where: ICCV
• Authors: Mingtao Pei, School of Computer Science, Yunde Jia, Song-Chun Zhu
• Keywords: Atomic Actions, Bayesian Framework, Computer Vision, ICCV 2011, Scsg