Real-Time 3D Reconstruction for Occlusion-Aware Interactions in Mixed Reality

In this paper, we present a system for performing real-time occlusion-aware interactions in a mixed reality environment. Our system consists of 16 ceiling-mounted cameras observing an interaction space of size 3.70 m x 3.20 m x 2.20 m. We reconstruct the shape of all objects inside the interaction space using a visual hull method at a frame rate of 30 Hz. Due to the interactive speed of the system, the users can act naturally in the interaction space. In addition, since we reconstruct the shape of every object, the users can use their entire body to interact with the virtual objects. This is a significant advantage over marker-based tracking systems, which require a prior setup and tedious calibration steps for every user who wants to use the system. With our system anybody can just enter the interaction space and start interacting naturally. We illustrate the usefulness of our system through two sample applications. The first application is a real-life version of the well known game...

• Type: Conference
• Year: 2009
• Where: ISVC
• Authors: Alexander Ladikos, Nassir Navab
• Keywords: Applied Computing, Interaction Space, ISVC 2009, Real-time Occlusion-aware Interactions, Virtual Objects