A new method for object search is proposed. The proposed scheme is based on matching gradient information around each pixel, computed in the form of orientation codes, rather than the gray levels directly and is robust against irregularities occurring in the real world scenes. A probabilistic model for robust matching is given and verified by real image data. Experimental results for real world scenes demonstrate the effectiveness of the proposed method for object search in the presence of different potential causes of mismatches.