Markerless reconstruction and synthesis of dynamic facial expressions

In this paper we combine methods from the field of computer vision with surface editing techniques to generate animated faces, which are all in full correspondence to each other. The inputs for our system are synchronized video streams from multiple cameras. The system produces a sequence of triangle meshes with fixed connectivity, representing the dynamics of the captured face. By carefully taking all requirements and characteristics into account we decided for the proposed system design: We deform an initial face template using movements estimated from the video streams. To increase the robustness of the reconstruction, we use a morphable model as a shape prior to initialize a surfel fitting technique which is able to precisely capture face shapes not included in the morphable model. In the deformation stage, we use a 2D mesh based tracking approach to establish correspondences over time. We then reconstruct positions in 3D using the same surfel fitting technique, and finally u...

• Type: Journal
• Year: 2011
• Where: CVIU
• Authors: Dominik Sibbing, Martin Habbecke, Leif Kobbelt
• Keywords: Cryptology, CVIU 2011, Face Shapes, Morphable Model, Triangle Meshes