An experimental evaluation of continuous testing during development

Continuous testing uses excess cycles on a developer’s workstation to continuously run regression tests in the background, providing rapid feedback about test failures as source code is edited. It is intended to reduce the time and energy required to keep code welltested and prevent regression errors from persisting uncaught for long periods of time. This paper reports on a controlled human experiment to evaluate whether students using continuous testing are more successful in completing programming assignments. We also summarize users’ subjective impressions and discuss why the results may generalize. The experiment indicates that the tool has a statistically significant effect on success in completing a programming task, but no such effect on time worked. Participants using continuous testing were three times as likely to complete the task before the deadline. Participants using continuous compilation were twice as likely to complete the task, providing empirical support to a co...

• Type: Conference
• Year: 2004
• Where: ISSTA
• Authors: David Saff, Michael D. Ernst
• Keywords: Continuous Compilation, Continuous Testing, ISSTA 2004, Keywords Continuous Testing