Crowdsourcing graphical perception: using mechanical turk to assess visualization design

Understanding perception is critical to effective visualization design. With its low cost and scalability, crowdsourcing presents an attractive option for evaluating the large design space of visualizations; however, it first requires validation. In this paper, we assess the viability of Amazon’s Mechanical Turk as a platform for graphical perception experiments. We replicate previous studies of spatial encoding and luminance contrast and compare our results. We also conduct new experiments on rectangular area perception (as in treemaps or cartograms) and on chart size and gridline spacing. Our results demonstrate that crowdsourced perception experiments are viable and contribute new insights for visualization design. Lastly, we report cost and performance data from our experiments and distill recommendations for the design of crowdsourced studies. ACM Classification: H5.2 [Information interfaces and presentation]: User Interfaces—Evaluation/Methodology General Terms: Experiment...

• Type: Conference
• Year: 2010
• Where: CHI
• Authors: Jeffrey Heer, Michael Bostock
• Keywords: CHI 2010, Effective Visualization Design, Human Computer Interaction, Rectangular Area Perception, Visualization Design