Various local descriptors have been used successfully in a variety of tasks including object recognition. Although different descriptors have been shown to have different strengths, they haven’t been used in combination. In this paper we show that by combining local image descriptors at the feature level, we can significantly improve object recognition performance. Our system uses keyed context patches and SIFT, two descriptors that have been shown to have a somewhat uncorrelated performance [9]. By requiring hypotheses generated by both types of descriptors to satisfy the same consistency constraints, we were able to significantly reduce the error rate on recognition and categorization tasks.