— In this paper we propose a method of realizing continuous tracking of a three-dimensional object by calculating moments of a translating and rotating object whose shape is known, either analytically or by using a table, and matching them with those of the input image. In simulation, the position and orientation is accurately recognized. Using a noise model and particle filter, we show that the position and orientation can be recognized even with noisy images.