Expert Search Evaluation by Supporting Documents

An expert search system assists users with their "expertise need" by suggesting people with relevant expertise to their query. Most systems work by ranking documents in response to the query, then ranking the candidates using information from this initial document ranking and known associations between documents and candidates. In this paper, we aim to determine whether we can approximate an evaluation of the expert search system using the underlying document ranking. We evaluate the accuracy of our document ranking evaluation by assessing how closely each measure correlates to the ground truth evaluation of the candidate ranking. Interestingly, we find that improving the underlying ranking of documents does not necessarily result in an improved candidate ranking.

• Type: Conference
• Year: 2008
• Where: ECIR
• Authors: Craig Macdonald, Iadh Ounis
• Keywords: Candidate Ranking, Document Ranking, Documents, ECIR 2008, Information Technology