Sciweavers

KDD
2012
ACM

Learning from crowds in the presence of schools of thought

12 years 1 months ago
Learning from crowds in the presence of schools of thought
Crowdsourcing has recently become popular among machine learning researchers and social scientists as an effective way to collect large-scale experimental data from distributed workers. To extract useful information from the cheap but potentially unreliable answers to tasks, a key problem is to identify reliable workers as well as unambiguous tasks. Although for objective tasks that have one correct answer per task, previous works can estimate worker reliability and task clarity based on the single gold standard assumption, for tasks that are subjective and accept multiple reasonable answers that workers may be grouped into, a phenomenon called schools of thought, existing models cannot be trivially applied. In this work, we present a statistical model to estimate worker reliability and task clarity without resorting to the single gold standard assumption. This is instantiated by explicitly characterizing the grouping behavior to form schools of thought with a rank-1 factorization of...
Yuandong Tian, Jun Zhu
Added 28 Sep 2012
Updated 28 Sep 2012
Type Journal
Year 2012
Where KDD
Authors Yuandong Tian, Jun Zhu
Comments (0)