There is considerable research in the field of content-based medical image retrieval; however, few of the current systems investigate the relationship between the radiologists’ visual impression of image similarity and the computer calculated content-based similarity. Furthermore, those research studies that investigate these relationships analyze the visual similarity with respect to degree of malignancy without including specific characteristics that are important in the diagnosis process. The creation of the NIH/NCI Lung Image Database Consortium (LIDC) dataset offers the opportunity to perform the proposed research. Each nodule out of the 932 distinct nodules (larger than 3mm in diameter) was delineated and annotated by up to four radiologists using nine semantic characteristics that are important in the lung nodule interpretation process. Using the LIDC images, we propose to encode the radiologists’ characteristic-based similarity and further discover if there is any relat...