This paper surveys the use of inferential statistics over the last two BCS HCI conferences and the last year (2006) of two leading HCI journals. Of the 80 papers covered, 41 used some form of inferential statistics. However, all but one had some form of problem of reporting or analysis that undermined the value or the validity of the statistical testing and hence the research findings. This paper discusses the implications of such widespread issues for HCI research and considers approaches for improving the use of statistics in HCI. Categories and Subject Descriptors A.1 Introductory and Survey General Terms Measurement, Reliability, Human Factors. Keywords Inferential statistics, reporting statistics, assumptions, overtesting, HCI, research methods
Paul A. Cairns