Relative effect of spam and irrelevant documents on user interaction with search engines

Meaningful evaluation of web search must take account of spam. Here we conduct a user experiment to investigate whether satisfaction with search engine result pages as a whole is harmed more by spam or by irrelevant documents. On some measures, search result pages are differentially harmed by the insertion of spam and irrelevant documents. Additionally we find that when users are given two documents of equal utility, the one with the lower spam score will be preferred; a result page without any spam documents will be preferred to one with spam; and an irrelevant document high in a result list is surprisingly more damaging to user satisfaction than a spam document. We conclude that web ranking and evaluation should consider both utility (relevance) and “spamminess” of documents. Categories and Subject Descriptors H.3.3 [Information Storage and Retrieval]: Information Search and Retrieval—Search process General Terms Measurement, Human Factors Keywords Web Search, Web Spam, Adve...

• Type: Journal
• Year: 2011
• Where: CIKM
• Authors: Timothy Jones, David Hawking, Paul Thomas, Ramesh Sankaranarayana
• Keywords: Adversarial Information Retrieval, CIKM 2011, Information Storage And Retrieval, Information Technology, Irrelevant Documents