: Because of the semantic gap between low-level feature and high-level semantic feature of images, the results of the traditional color-based image retrieval can't meet users' needs. In order to eliminate interference factors in the image retrieval, use image semantic feature and improve the accuracy of image retrieval, the paper introduces an algorithm based on the color correlation mining. It regards the pixel rows as a transaction set, uses the Apriori algorithm to find out the rows by looking for the continual co-occurrence color in the transaction set. These rows are correlative with the semantic object of the image. Then it extracts the correlative color histogram of image form the correlative color set to realize the correlative color mining.