It's just a matter of perspective(s): Crowd-Powered Consensus Organization of Corpora

We study the problem of organizing a collection of objects—images, videos—into clusters, using crowdsourcing. This problem is notoriously hard for computers to do automatically, and even with crowd workers, is challenging to orchestrate: (a) workers may cluster based on different latent hierarchies or perspectives; (b) workers may cluster at different granularities even when clustering using the same perspective; and (c) workers may only see a small portion of the objects when deciding how to cluster them (and therefore have limited understanding of the “big picture”). We develop costefficient, accurate algorithms for identifying the consensus organization (i.e., the organizing perspective most workers prefer to employ), and incorporate these algorithms into a cost-effective workflow for organizing a collection of objects, termed ORCHESTRA. We compare our algorithms with other algorithms for clustering, on a variety of real-world datasets, and demonstrate that ORCHESTRA orga...

• Type: Journal
• Year: 2016
• Where: CORR
• Authors: Ayush Jain, Joon Young Seo, Karan Goel, Andrew Kuznetsov, Aditya G. Parameswaran, Hari Sundaram
• Keywords: CORR 2016, Education