Free viewpoint video extraction, representation, coding, and rendering

Free viewpoint video provides the possibility to freely navigate within dynamic real world video scenes by choosing arbitrary viewpoints and view directions. So far, related work only considered free viewpoint video extraction, representation, and rendering methods. Compression and transmission has not yet been studied in detail and combined with the other components into one complete system. In this paper, we present such a complete system for efficient free viewpoint video extraction, representation, coding, and interactive rendering. Data representation is based on 3D mesh models and viewdependent texture mapping using video textures. The geometry extraction is based on a shape-from-silhouette algorithm. The resulting voxel models are converted into 3D meshes that are coded using MPEG-4 SNHC tools. The corresponding video textures are coded using an H.264/AVC codec. Our algorithms for view-dependent texture mapping have been adopted as an extension of MPEG-4 AFX. The presented resu...

• Type: Conference
• Year: 2004
• Where: ICIP
• Authors: Aljoscha Smolic, Karsten Müller, Matthias Kautzner, Peter Eisert, Philipp Merkle, Thomas Wiegand, Tobias Rein
• Keywords: Corresponding Video Textures, Efficient Free Viewpoint, Free Viewpoint Video, ICIP 2004, Image Processing, Viewpoint Video Extraction, World Video Scenes