Back to the Roots: Mean-Variance Analysis of Relevance Estimations

Recently, mean-variance analysis has been proposed as a novel paradigm to model document ranking in Information Retrieval. The main merit of this approach is that it diversifies the ranking of retrieved documents. In its original formulation, the strategy considers both the mean of relevance estimates of retrieved documents and their variance. However, when this strategy has been empirically instantiated, the concepts of mean and variance are discarded in favour of a point-wise estimation of relevance (to replace the mean) and of a parameter to be tuned or, alternatively, a quantity dependent upon the document length (to replace the variance). In this paper we revisit this ranking strategy by going back to its roots: mean and variance. For each retrieved document, we infer a relevance distribution from a series of point-wise relevance estimations provided by a number of different systems. This is used to compute the mean and the variance of document relevance estimates. On the TREC C...

• Type: Journal
• Year: 2011
• Where: ECIR
• Authors: Guido Zuccon, Leif Azzopardi, Keith van Rijsbergen
• Keywords: Document Length, Document Relevance, ECIR 2011, Information Technology, Variance Analysis