In this paper, a novel feature selection algorithm for object tracking is proposed. This algorithm performs more robust than the previous works by taking the correlation between features into consideration. Pixels of object/background regions are first treated as training samples. The feature selection problem is then modeled as finding a good subset of features and constructing a compound likelihood image with better discriminability for the tracking process. By adopting the AdaBoost algorithm, we iteratively select one best feature which compensate the previous selected features and linearly combine the set of corresponding likelihood images to obtain the compound likelihood image. We include the proposed algorithm into the mean shift based tracking system. Experimental results demonstrate that the proposed algorithm achieve very promising results.