We show that the discrimination between visually similar classes often depends on the detection of socalled ‘satellite features’. These are local features which are not informative by themselves, and can only be detected reliably at locations specified relative to other features. This makes satellite features difficult to extract by current classification methods. We describe a novel scheme which can extract discriminative satellite features and use them to distinguish between visually similar classes. The algorithm first searches for a set of features (“anchor features”) that can be found in all the similar classes. Such features can be detected because the classes are visually similar. The anchors are used to determine the locations of satellite features, which are extracted during learning and used in classification to distinguish between the similar classes. The algorithm is fully automatic, and is shown to work well for many categories of visually similar classes.