Active query selection for learning rankers

Methods that reduce the amount of labeled data needed for training have focused more on selecting which documents to label than on which queries should be labeled. One exception to this [4] uses expected loss optimization (ELO) to estimate which queries should be selected but is limited to rankers that predict absolute graded relevance. In this work, we demonstrate how to easily adapt ELO to work with any ranker and show that estimating expected loss in DCG is more robust than NDCG even when the final performance measure is NDCG. Categories and Subject Descriptors H.3.3 [Information Search and Retrieval]: Information Search and Retrieval—active learning Keywords Active learning, query selection

• Type: Journal
• Year: 2012
• Where: SIGIR
• Authors: Mustafa Bilgic, Paul N. Bennett
• Keywords: Dcg, Information Technology, Query Selection, SIGIR 2012, Subject Descriptors