Abstract. An approach for visual information analysis and classification is presented. It is based on a knowledge synthesizing technique to automatically create a relevance map from essential areas in natural images. It also derives a set of well-structured representations from low-level description to drive the final classification. The backbone of this approach is a distribution mapping strategy involving a knowledge synthesizing module based on an intelligent growing when required network. Classification is achieved by simulating the high-level top-down visual information perception in primates followed by incremental Bayesian parameter estimation. The proposed modular system architecture offers straightforward expansion to include user relevance feedback, contextual input, and multimodal information if available.