Video segmentation by tracing discontinuities in a trajectory embedding

Our goal is to segment a video sequence into moving objects and the world scene. In recent work, spectral embedding of point trajectories based on 2D motion cues accumulated from their lifespans, has shown to outperform factorization and per frame segmentation methods for video segmentation. The scale and kinematic nature of the moving objects and the background scene determine how close or far apart trajectories are placed in the spectral embedding. Such density variations may confuse clustering algorithms, causing over-fragmentation of object interiors. Therefore, instead of clustering in the spectral embedding, we propose detecting discontinuities of embedding density between spatially neighboring trajectories. Detected discontinuities are strong indicators of object boundaries and thus valuable for video segmentation. We propose a novel embedding discretization process that recovers from over-fragmentations by merging clusters according to discontinuity evidence along inter-cluste...

• Type: Journal
• Year: 2012
• Where: CVPR
• Authors: Katerina Fragkiadaki, Geng Zhang, Jianbo Shi
• Keywords: Computer Vision, Cvpr 2012, Density Variations, Saliency Maps, Video Segmentation