— Non-rigid object detection is a challenging open research problem in computer vision. It is a critical part in many applications such as image search, surveillance, humancomputer interaction or image auto-annotation. Most successful approaches to non-rigid object detection make use of partbased models. In particular, Conditional Random Fields (CRF) have been successfully embedded into a discriminative partsbased model framework due to its effectiveness for learning and inference (usually based on a tree structure). However, CRFbased approaches do not incorporate global constraints and only model pairwise interactions. This is especially important when modeling object classes that may have complex parts interactions (e.g. facial features or body articulations), because neglecting them yields an oversimplified model with suboptimal performance. To overcome this limitation, this paper proposes a novel hierarchical CRF (HCRF). The main contribution is to build a hierarchy of part comb...