Crowdsourced Databases: Query Processing with People

Amazon’s Mechanical Turk (“MTurk”) service allows users to post short tasks (“HITs”) that other users can receive a small amount of money for completing. Common tasks on the system include labelling a collection of images, combining two sets of images to identify people which appear in both, or extracting sentiment from a corpus of text snippets. Designing a workflow of various kinds of HITs for filtering, aggregating, sorting, and joining data sources together is common, and comes with a set of challenges in optimizing the cost per HIT, the overall time to task completion, and the accuracy of MTurk results. We propose Qurk, a novel query system for managing these workflows, allowing MTurk-style processing of relational databases. We describe a number of query execution and optimization challenges, and discuss some potential solutions.

• Type: Journal
• Year: 2011
• Where: CIDR
• Authors: Adam Marcus 0002, Eugene Wu 0002, Samuel Madden, Robert C. Miller
• Keywords: Algorithms, CIDR 2011, Mechanical Turk, Query Execution, Text Snippets