Variation of Relevance Assessments for Medical Image Retrieval

Abstract. Evaluation is crucial for the success of most research domains, and image retrieval is no exception to this. Recently, several benchmarks have been developed for visual information retrieval such as TRECVID, ImageCLEF, and ImagEval to create frameworks for evaluating image retrieval research. An important part of evaluation is the creation of a ground truth or gold standard to evaluate systems against. Much experience has been gained on creating ground truths for textual information retrieval, but for image retrieval these issues require further research. This article will present the process of generating relevance judgements for the medical image retrieval task of ImageCLEF. Many of the problems encountered can be generalised to other image retrieval tasks as well, so the outcome is not limited to the medical domain. Part of the images analysed for relevance were judged by two assessors, and these are analysed with respect to their consistency and potential problems. Our go...

• Type: Conference
• Year: 2006
• Where: AMR
• Authors: Henning Müller, Paul Clough, William R. Hersh, Antoine Geissbühler
• Keywords: AMR 2006, Image Retrieval, Image Retrieval Tasks, Information Retrieval, Multimedia