A key point in automatic age estimation is to design feature set essential to age perception. To achieve this goal, this paper builds up a hierarchical graphical face model for faces appearing at low, middle and high resolution respectively. Along the hierarchy, a face image is decomposed into detailed parts from coarse to fine. Then four types of features are extracted from this graph representation guided by the priors of aging process embedded in the graphical model: topology, geometry, photometry and configuration. On age estimation, this paper follows the popular regression formulation for mapping feature vectors to its age label. The effectiveness of the presented feature set is justified by testing results on two datasets using different kinds of regression methods. The experimental results in this paper show that designing feature set for age estimation under the guidance of hierarchical face model is a promising method and a flexible framework as well.