A head pose estimation system is described, which uses low resolution video sequences to determine the orientation and position of a head with respect to a internally calibrated camera. The system employs a feature based approach to roughly estimate the head pose and an approach using a symmetry based illumination model to refine the head pose independent of the users albedo and illumination influences.