This paper introduces a composite learning approach for image retrieval with relevance feedback. The proposed system combines the radial basis function (RBF) based lowlevel learning and the semantic learning space (SLS) based high-level learning to retrieve the desired images with fewer than 3 feedback steps. User’s relevance feedback is utilized for updating both low-level and high-level features of the query image. Specifically, the RBF-based learning captures the non-linear relationship between the low-level features and the semantic meaning of an image. The SLS-based learning stores semantic features of each database image using randomly chosen semantic basis images. The similarity score is computed as the weighted combination of normalized similarity scores yielded from both RBF and SLS learning. Extensive experiments evaluate the performance of the proposed approach and demonstrate our system achieves higher retrieval accuracy than peer systems.