TurKit: human computation algorithms on mechanical turk

Mechanical Turk provides an on-demand source of human computation. This provides a tremendous opportunity to explore algorithms which incorporate human computation as a function call. However, various systems challenges make this difficult in practice, and most uses of Mechanical Turk post large numbers of independent tasks. TurKit is a toolkit for prototyping and exploring truly algorithmic human computation, while maintaining a straight-forward imperative programming style. We present the crash-andrerun programming model that makes TurKit possible, along with a variety of applications for human computation algorithms. We also present a couple case studies of TurKit used for real experiments outside our lab. ACM Classification: H5.2 [Information interfaces and presentation]: User Interfaces. - Prototyping. General terms: Algorithms, Design, Experimentation

• Type: Journal
• Year: 2010
• Where: UIST
• Authors: Greg Little, Lydia B. Chilton, Max Goldman, Robert C. Miller
• Keywords: Human Computation, Human Computation Algorithms, Mechanical Turk, Software Engineering, UIST 2010