Unknown Rewards in Finite-Horizon Domains

"Human computation" is a recent approach that extracts information from large numbers of Web users. reCAPTCHA is a human computation project that improves the process of digitizing books by getting humans to read words that are difficult for OCR algorithms to read (von Ahn et al. 2008). In this paper, we address an interesting strategic control problem inspired by the reCAPTCHA project: given a large set of words to transcribe within a time deadline, how can we choose the difficulty level such that we maximize the probability of successfully transcribing a document on time? Our approach is inspired by previous work on timed, zero-sum games, as we face an analogous timed policy decision on the choice of words to present to users. However, our Web-based word transcribing domain is particularly challenging as the reward of the actions is not known; i.e., there is no knowledge if the spelling provided by a human is actually correct. We contribute an approach to solve this proble...

• Type: Conference
• Year: 2008
• Where: AAAI
• Authors: Colin McMillen, Manuela M. Veloso
• Keywords: AAAI 2008, Human Computation, Human Computation Project, Intelligent Agents, ReCAPTCHA Project