Average task times in usability tests: what to report?

The distribution of task time data in usability studies is positively skewed. Practitioners who are aware of this positive skew tend to report the sample median. Monte Carlo simulations using data from 61 large-sample usability tasks showed that the sample median is a biased estimate of the population median. Using the geometric mean to estimate the center of the population will, on average, have 13% less error and 22% less bias than the sample median. Other estimates of the population center (trimmed, harmonic and Winsorized means) had worse performance than the sample median. Author Keywords Usability evaluation, task times, median, geometric mean, Monte Carlo simulations ACM Classification Keywords H5.m. Information interfaces and presentation (e.g., HCI): User Interfaces–Evaluation/Methodology. General Terms Measurement, Human Factors, Experimentation

• Type: Conference
• Year: 2010
• Where: CHI
• Authors: Jeff Sauro, James R. Lewis
• Keywords: CHI 2010, Human Computer Interaction, Monte Carlo Simulations, Population Median, Sample Median