Domain bias in web search

This paper uncovers a new phenomenon in web search that we call domain bias — a user’s propensity to believe that a page is more relevant just because it comes from a particular domain. We provide evidence of the existence of domain bias in click activity as well as in human judgments via a comprehensive collection of experiments. We begin by studying the difference between domains that a search engine surfaces and that users click. Surprisingly, we find that despite changes in the overall distribution of surfaced domains, there has not been a comparable shift in the distribution of clicked domains. Users seem to have learned the landscape of the internet and their click behavior has thus become more predictable over time. Next, we run a blind domain test, akin to a Pepsi/Coke taste test, to determine whether domains can shift a user’s opinion of which page is more relevant. We find that domains can actually flip a user’s preference about 25% of the time. Finally, we demon...

• Type: Journal
• Year: 2012
• Where: WSDM
• Authors: Samuel Ieong, Nina Mishra, Eldar Sadikov, Li Zhang
• Keywords: Data Mining, Domain Test, Human Judgments, Pepsi Coke, WSDM 2012