We present a new active active recognition scheme (using an uncalibrated camera) based on a new idea, appearancebased aspect graphs. The scheme is robust to background clutter, and affine transformations of the object. We use a probabilistic reasoning framework which helps in probability calculations and planning the next view (when a view of the object does not contain sufficient features to recognise it unambiguously), in conjunction with a new hierarchical knowledge representation scheme. Preliminary experiments with the system show encouraging results.