Abstract-- This paper describes a framework for constructing a three-dimensional immersive environment that can be used for training physical activities. The system is designed to capture three dimensional full-body human motion in real time and visualize the data either locally or remotely through three-dimensional display system so that the data can be viewed from arbitrary viewpoints. In the proposed system, an immersive environment is constructed through realistic reconstruction of a scene by applying a stereo algorithm on a set of images that is captured from multiple viewpoints. Specifically, twelve camera clusters that consist of four camera quadruples are used to capture the scene, where each camera cluster is processed by a pc independently and synchronously so that partial reconstructions from each viewpoint are merged to form a complete 3D description of the scene. This paper discusses in detail system architectures that enable synchronous operations across multiple computer...