Thumbs-Up: a game for playing to rank search results

Human computation is an effective way to channel human effort spent playing games to solving computational problems that are easy for humans but difficult for computers to automate. We propose Thumbs-Up, a new game for human computation with the purpose of playing to rank search result. Our experience from users shows that Thumbs-Up is not only fun to play, but produces more relevant rankings than both a major search engine and optimal rank aggregation using the Kemeny rule. Categories and Subject Descriptors H.3.3 [Information Search and Retrieval]: Search Process; I.2.6 [Learning]: Knowledge Acquisition General Terms Design, Experimentation, Human Factors, Measurement Keywords Games with a purpose, human computation, online games, rank aggregation, relevance, search engine

• Type: Conference
• Year: 2009
• Where: KDD
• Authors: Ali Dasdan, Chris Drome, Santanu Kolay, Micah Alpern, Alice Han, Tom Chi, Jamie Hoover, Ivan Davtchev, Sharad Verma
• Keywords: Data Mining, Human Computation, KDD 2009, Rank Aggregation, Search Engine