On the Predictability of Random Tests for Object-Oriented Software

Intuition suggests that random testing of object-oriented programs should exhibit a high difference in the number of defects detected by two different runs over the same amount of time. As a consequence, random testing would be rather unpredictable. This article evaluates the variance of the number of defects detected by random testing over time. We present the results of an empirical study that is based on 1215 hours of randomly testing 27 Eiffel classes for 30 seeds each. Analyzing the over 6 million errors triggered during the experiments, the study provides evidence that the relative number of defects detected by random testing over time is predictable and that different runs of the random test case generator detect largely different defects. The study also shows that random testing quickly finds defects: the first defect is very likely to be found within 30 seconds.

• Type: Conference
• Year: 2008
• Where: ICST
• Authors: Ilinca Ciupa, Alexander Pretschner, Andreas Leitner, Manuel Oriol, Bertrand Meyer
• Keywords: ICST 2008, Random Test Case, Random Testing, Software Engineering, Study Provides Evidence