In this paper, we introduce a new approach for modeling
visual context. For this purpose, we consider the leaves of a
hierarchical segmentation tree as elementary units. Each
leaf is described by features of its ancestral set, the re-
gions on the path linking the leaf to the root. We con-
struct region trees by using a high-performance segmen-
tation method. We then learn the importance of different
descriptors (e.g. color, texture, shape) of the ancestors for
classification. We report competitive results on the MSRC
segmentation dataset and the MIT scene dataset, showing
that region ancestry efficiently encodes information about
discriminative parts, objects and scenes.
Joseph J. Lim, Pablo Arbelaez, Chunhui Gu, and Jit