In this paper, we draw an analogy between image retrieval and text retrieval and propose a visual phrase-based approach to retrieve images containing desired objects. The visual phrase is defined as a pair of adjacent local image patches and is constructed using data mining. We devise methods on how to construct visual phrases from images and how to encode the visual phrase for indexing and retrieval. Our experiments demonstrate that visual phrase-based retrieval approach can be very efficient and can be 20% more effective than its visual wordbased counterpart. Categories and Subject Descriptors H.2.8 [Database Management]: Database Applications – Image databases. H.3.1 [Information Storage and Retrieval]: Content Analysis and Indexing – Indexing methods; General Terms Algorithms, Experimentation, Performance. Keywords Object-based image retrieval, visual phrase, SIFT, inverted index.