The covariance region descriptor recently proposed in [1] has been proved robust and versatile for a modest computational cost. The covariance matrix enables efficient fusion of different types of features, where the spatial and statistical properties as well as their correlation are characterized. The similarity of two covariance descriptor is measured on Riemannian manifolds. Relying on the same metric, but within a probabilistic framework, we propose a novel tracking approach on Riemannian manifolds. The particle filtering technique allows us to better handle background clutter, as well as the temporary occlusions of the target. Furthermore, we extend the fast covariance computation to the tracking problem, which makes the tracking procedure more efficient. The proposed approach is robust to noises and much faster than the original search-based covariance tracker [2]. Extensive experimental results demonstrate greatly improved performance over classical color-based Bayesian tracker...