Dense 3D Motion Capture for Human Faces

This paper proposes a novel approach to motion capture from multiple, synchronized video streams, specifically aimed at recording dense and accurate models of the structure and motion of highly deformable surfaces such as skin, that stretches, shrinks, and shears in the midst of normal facial expressions. Solving this problem is a key step toward effective performance capture for the entertainment industry, but progress so far has been hampered by the lack of appropriate local motion and smoothness models. The main technical contribution of this paper is a novel approach to regularization adapted to nonrigid tangential deformations. Concretely, we estimate the nonrigid deformation parameters at each vertex of a surface mesh, smooth them over a local neighborhood for robustness, and use them to regularize the tangential motion estimation. To demonstrate the power of the proposed approach, we have integrated it into our previous work for markerless motion capture [9], an...

• Type: Conference
• Year: 2009
• Where: CVPR
• Authors: Yasutaka Furukawa (University of Washington), Jean Ponce (Ecole Normale Superieure)
• Keywords: Computer Vision, CVPR 2009, Dense 3D Motion, Synchronized Video Streams