We have developed a method for recognizing and tracking known three-dimensional objects in scene images captured from a user’s viewpoint. The method uses multiple color histogram matching and condensation tracking. We implemented a prototype 3-D online manual system based on our method.