Abstract. We present a biologically inspired vision system able to incrementally learn multiple visual categories by interactively presenting several hand-held objects. The overall system is composed of a foregroundbackground separation part, several feature extraction methods and a life-long learning approach combining incremental learning with category specific feature selection. In contrast to most visual categorization approaches where typically each view is assigned to a single category we allow labeling with an arbitrary number of shape and color categories and also impose no restrictions to the viewing angle of presented objects.