Stereo Matching with Reflections and Translucency

In this paper, we address the stereo matching problem in the presence of reflections and translucency, where image formation can be modeled as the additive superposition of layersatdifferentdepth. Thepresenceofsucheffectsviolates the Lambertian assumption underlying traditional stereo vision algorithms, making it impossible to recover component depths using direct color matching based methods. We develop several techniques to estimate both depths and colors of the component layers. Depth hypotheses are enumerated in pairs, one from each layer, in a nested plane sweep. For each pair of depth hypotheses, we compute a componentcolor-independent matching error per pixel, using a spatialtemporal-differencing technique. We then use graph cut optimization to solve for the depths of both layers. This is followed by an iterative color update algorithm whose convergence is proven in our paper. We show convincing results of depth and color estimates for both synthetic and real image sequences.

• Type: Conference
• Year: 2003
• Where: CVPR
• Authors: Yanghai Tsin, Sing Bing Kang, Richard Szeliski
• Keywords: Component Depths, Componentcolor-independent Matching Error, Computer Vision, CVPR 2003, Depth Hypotheses, Iterative Color Update, Stereo Matching Problem