In this paper, we formulate the shape localization problem in the Bayesian framework. In the learning stage, we propose the Constrained RankBoost approach to model the likelihood of local features associated with the key points of an object, like face, while preserve the prior ranking order between the ground truth position of a key point and its neighbors; in the inferring stage, a simple efficient iterative algorithm is proposed to uncover the MAP shape by locally modeling the likelihood distribution around each key point via our proposed variational Locally Weighted Learning (VLWL) method. Our proposed framework has the following benefits: 1) compared to the classical PCA models, the likelihood presented by the ranking prior likelihood model has more discriminating power as to the optimal position and its neighbors, especially in the problem with ambiguity between the optimal positions and their neighbors; 2) the VLWL method guarantees that the posterior probability of the derived ...