The paper proposes a method to keep the tracker robust to background clutters by online selecting discriminative features from a large feature space. Furthermore, the feature selection procedure is embedded into the particle filtering process with the aid of existed “background” particles. Feature values from background patches and object observations are sampled during tracking and Fisher discriminant is employed to rank the classification capacity of each feature based on sampled values. Top-ranked discriminative features are selected into the appearance model and simultaneously invalid features are removed out to adjust the object representation adaptively. The implemented tracker with online discriminative feature selection module embedded shows promising results on experimental video sequences.