Video Scene Understanding Using Multi-scale Analysis

We propose a novel method for automatically discover-ing key motion patterns happening in a scene by observing the scene for an extended period. Our method does not rely on object detection and tracking, and uses low level fea-tures, the direction of pixel wise optical flow. We first di-vide the video into clips and estimate a sequence of flow-fields. Each moving pixel is quantized based on its location and motion direction. This is essentially a bag of words representation of clips. Once a bag of words repre-sentation is obtained, we proceed to the screening stage, using a measure called the ‘conditional entropy’. After obtaining useful words we apply Diffusion maps. Diffusion maps framework embeds the manifold points into a lower dimensional space while preserving the intrinsic local geometric structure. Finally, these useful words in lower dimensional space are clustered to discover key motion patterns. Diffusion map embedding involves diffusion time parameter which gives us abi...

• Type: Conference
• Year: 2009
• Where: ICCV
• Authors: Yang Yang, Jingen Liu, Mubarak Shah
• Keywords: Computer Vision, ICCV 2009, Video Scene Understanding