Using RankBoost to compare retrieval systems

This paper presents a new pooling method for constructing the assessment sets used in the evaluation of retrieval systems. Our proposal is based on RankBoost, a machine learning voting algorithm. It leads to smaller pools than classical pooling and thus reduces the manual assessment workload for building test collections. Experimental results obtained on an XML document collection demonstrate the effectiveness of the approach according to different evaluation criteria. Categories and Subject Descriptors H.3.3 [Information Search and Retrieval]: Retrieval Models; I.2.6 [Artificial Intelligence]: Learning—parameter learning General Terms Algorithms, Experimentation, Measurement Keywords Pooling, RankBoost, XML retrieval evaluation

• Type: Conference
• Year: 2005
• Where: CIKM
• Authors: Huyen-Trang Vu, Patrick Gallinari
• Keywords: CIKM 2005, Information Management, Manual Assessment Workload, Retrieval Systems, XML Retrieval Evaluation