Markerless Motion Capture of Interacting Characters Using Multi-view Image Segmentation

We present a markerless motion capture approach that reconstructs the skeletal motion and detailed time-varying surface geometry of two closely interacting people from multi-view video. Due to ambiguities in feature-to-person assignments and frequent occlusions, it is not feasible to directly apply single-person capture approaches to the multiperson case. We therefore propose a combined image segmentation and tracking approach to overcome these difficulties. A new probabilistic shape and appearance model is employed to segment the input images and to assign each pixel uniquely to one person. Thereafter, a single-person markerless motion and surface capture approach can be applied to each individual, either one-by-one or in parallel, even under strong occlusions. We demonstrate the performance of our approach on several challenging multi-person motions, including dance and martial arts, and also provide a reference dataset for multi-person motion capture with ground truth.

• Type: Journal
• Year: 2011
• Where: CVPR
• Authors: Yebin Liu, Carsten Stoll, Juergen Gall, Hans-Peter Seidel, Christian Theobalt
• Keywords: Computer Vision, CVPR 2011, Markerless Motion, Multi-person Motion, Multi-person Motion Capture