An empirical study of the robustness of MacOS applications using random testing

We report on the fourth in a series of studies on the reliability of application programs in the face of random input. Over the previous 15 years, we have studied the reliability of UNIX command line and X-Window based (GUI) applications and Windows applications. In this study, we apply our fuzz testing techniques to applications running on the Mac OS X operating system. We continue to use a simple, or even simplistic technique: unstructured black-box random testing, considering a failure to be a crash or hang. As in the previous three studies, the technique is crude but seems to be effective in locating bugs in real programs. We tested the reliability of 135 command-line UNIX utilities and thirty graphical applications on Mac OS X by feeding random input to each. We report on application failures – crashes (dumps core) or hangs (loops indefinitely) – and, where source code is available, we identify the causes of these failures and categorize them. Our testing crashed only 7% of ...

• Type: Conference
• Year: 2006
• Where: ISSTA
• Authors: Barton P. Miller, Gregory Cooksey, Fredrick Moore
• Keywords: Black-box Random Testing, ISSTA 2006, Random Input, Random Testing