A simple free viewpoint video system which is able not only to display user-specified views at arbitrary angle but also to efficiently stream the necessary video over a network is described. Virtual view synthesis based on projection theory was used to obtain intermediate views between multiple cameras. The depth information, which is required for the virtual view synthesis, was obtained using the segmentation-based stereo matching algorithm. For real-time rendering, our system was optimized using Single Instructions, Multiple Data (SIMD) technology. For efficient streaming, a novel method of combining the video with the depth information is proposed.
Seokhwan Jo, Dohyun Lee, Yoonseob Kim, Chang D. Yo