Current Web search tools do a good job of retrieving documents that satisfy the wide range of intentions that people associate with a query – but do not do a very good job of discerning different individuals’ unique search goals. In this paper we explore the variation in what different people consider relevant to the same query by mining three data sources: 1) explicit relevance judgments, 2) clicks on search results (a behavior-based implicit measure of relevance), and 3) the similarity of desktop content to search results (a content-based implicit measure of relevance). We find that people’s explicit judgments for the same queries differ greatly. As a result, there is a large gap between how well search engines could perform if they were to tailor results to the individual, and how well they currently perform by returning results designed to satisfy everyone. We call this gap the potential for personalization. The two implicit indicators we studied provide complimentary value ...
Jaime Teevan, Susan T. Dumais, Eric Horvitz