In this paper, we propose a new method for object tracking based on mean shift algorithm using a kernel which has the shape of the target object, and with probabilistic estimation of the orientation change and scale adaptation. The proposed method uses an object mask to construct a kernel which has the shape of the actual object for tracking. Orientation is adjusted using probabilistic estimation of orientation and scale is adapted using a newly proposed descriptor for scale. Tests results show that the proposed method is robust to background clutter and tracks objects very accurately.