In this paper, we present a novel methodology to detect and recognize objects in cluttered scenes by proposing boosted contextual descriptors of landmarks in a framework of multi-class object recognition. To detect a sample of the object class, Boosted Landmarks identify landmark candidates in the image and define a constellation of contextual descriptors able to capture the spatial relationship among them. To classify the object, we consider the problem of multi-class classification with a battery of classifiers trained to share their knowledge among classes. For this purpose, we extend the Error Correcting Output Codes technique proposing a methodology based on embedding a forest of optimal tree structures. We validated our approach using public data-sets from the UCI and Caltech databases. Furthermore, we show results of the technique applied to a real computer vision problem: detection and categorization of traffic signs. Ó 2007 Elsevier B.V. All rights reserved.