This paper investigates the agreement of relevance assessments between official TREC judgments and those generated from an interactive IR experiment. Results show that 63% of docu...
Assessors frequently disagree on the topical relevance of documents. How much of this disagreement is due to ambiguity in assessment instructions? We have two assessors assess TRE...
Information retrieval systems are evaluated against test collections of topics, documents, and assessments of which documents are relevant to which topics. Documents are chosen fo...
Finding relevant information on the Web can be a complex task for most of the users. Although Web search applications are improving, they still need to be more intelligent to adapt...
This paper provides an overview of the newly launched Book Search Track at INEX 2007 (BookSearch’07), its participants, tasks, book corpus, test topics and relevance assessments...