Benchmarking image and video retrieval: an overview

Multimedia Information Retrieval (IR) techniques and associated systems are now numerous and justify the development of strategies and actions to objectively evaluate their capabilities. A number of initiatives following this line exist, each with its own peculiarities. In this paper, we take a bird’s eye view on benchmarking multimedia IR systems (with the particular case of image and video retrieval) and summarize contributions made to a dedicated special session at the ACM Multimedia Information Retrieval Workshop (ACM MIR2006). From the analysis of a classical IR system, we identify locations of interest for evaluation of performance. We review proposals made in the context of existing benchmarks, each specialized in its own aspect and media. Categories and Subject Descriptors H.3.3 [Information search and retrieval]: Search process; H.3.1 [Content Analysis and Indexing]: Indexing methods; H.5.1 [Multimedia Information systems]: Evaluation/Methodology Keywords Evaluation, multim...

• Type: Conference
• Year: 2006
• Where: MIR
• Authors: Stéphane Marchand-Maillet, Marcel Worring
• Keywords: Ir System, MIR 2006, Multimedia, Multimedia Information Retrieval, Multimedia Ir Systems