A Comparison of Interactive and Ad-Hoc Relevance Assessments

Abstract. In this paper we report an initial comparison of relevance assessments made as part of the INEX 2006 Interactive Track (itrack’06) to those made for the topic assessment phase of the INEX 2007 ad-hoc track. The results indicate that that there are important differences in what information was assessed under the two different conditions, but it also suggests a certain level of agreement in what constitutes relevant and non-relevant information. In addition, there are indications that the task type has an influence on the distribution of relevance assessments.

• Type: Conference
• Year: 2007
• Where: INEX
• Authors: Birger Larsen, Saadia Malik, Anastasios Tombros
• Keywords: INEX, INEX 2007, Information Management, Relevance Assessments, Topic Assessment Phase