— This paper proposes a tracking system called VooDoo for 3d tracking of human body movements based on a 3d body model and the Iterative Closest Point (ICP) algorithm. The proposed approach is able to incorporate raw data from different input sensors, as well as results from feature trackers in 2d or 3d. All input data is processed within the same model fitting step by modeling all input measurements in 3d model space. The system has been implemented and runs in realtime at appr. 1014 Hz. Experiments with complex human movements exhibit the characteristics and advantages of the proposed approach.