A head tracking system for automatically detecting and tracking human heads in complex backgrounds is developed. In this paper, two issues are addressed: the detection ofhumanheadsandthedevelopmentofaheadtrackingsystem. First,basedonanelliptical model for the human head, we propose a Maximum Likelihood (ML) detector to reliably locate human heads in images having complex backgrounds. This ellipsebased ML head detector is relatively insensitive to illumination and rotation of the human heads, and its computation is similar to template matching. Second, we develop a head tracking system that can monitor the entrance of a person, detect and track the person's head, and then control the stereo cameras to focus their gaze on this person's head. Difference images are used to detect the entrance of a human. The ellipse-based ML head detector and the mutually-supported constraint are used to extract the corresponding ellipses in a stereo image pair. Then, the 3D position computed fr...