Directly optimizing evaluation measures in learning to rank

One of the central issues in learning to rank for information retrieval is to develop algorithms that construct ranking models by directly optimizing evaluation measures used in information retrieval such as Mean Average Precision (MAP) and Normalized Discounted Cumulative Gain (NDCG). Several such algorithms including SVMmap and AdaRank have been proposed and their effectiveness has been verified. However, the relationships between the algorithms are not clear, and furthermore no comparisons have been conducted between them. In this paper, we conduct a study on the approach of directly optimizing evaluation measures in learning to rank for Information Retrieval (IR). We focus on the methods that minimize loss functions upper bounding the basic loss function defined on the IR measures. We first provide a general framework for the study and analyze the existing algorithms of SVMmap and AdaRank within the framework. The framework is based on upper bound analysis and two types of upper b...

• Type: Journal
• Year: 2008
• Where: SIGIR
• Authors: Jun Xu, Tie-Yan Liu, Min Lu, Hang Li, Wei-Ying Ma
• Keywords: Algorithms, Evaluation Measure, Information Retrieval, Information Technology, SIGIR 2008