BiDi screen: a thin, depth-sensing LCD for 3D interaction using light fields

We transform an LCD into a display that supports both 2D multitouch and unencumbered 3D gestures. Our BiDirectional (BiDi) screen, capable of both image capture and display, is inspired by emerging LCDs that use embedded optical sensors to detect multiple points of contact. Our key contribution is to exploit the spatial light modulation capability of LCDs to allow lensless imaging without interfering with display functionality. We switch between a display mode showing traditional graphics and a capture mode in which the backlight is disabled and the LCD displays a pinhole array or an equivalent tiled-broadband code. A large-format image sensor is placed slightly behind the liquid crystal layer. Together, the image sensor and LCD form a mask-based light field camera, capturing an array of images equivalent to that produced by a camera array spanning the display surface. The recovered multi-view orthographic imagery is used to passively estimate the depth of scene points. Two motivatin...

• Type: Conference
• Year: 2009
• Where: SIGGRAPH
• Authors: Matthew Hirsch, Douglas Lanman, Henry Holtzman, Ramesh Raskar
• Keywords: Computer Graphics, Display Mode, Image Sensor, Large-format Image Sensor, SIGGRAPH 2009