Test coverage and post-verification defects: A multiple case study

Test coverage is a promising measure of test effectiveness and development organizations are interested in costeffective levels of coverage that provide sufficient fault removal with contained testing effort. We have conducted a multiple-case study on two dissimilar industrial software projects to investigate if test coverage reflects test effectiveness and to find the relationship between test effort and the level of test coverage. We find that in both projects the increase in test coverage is associated with decrease in field reported problems when adjusted for the number of prerelease changes. A qualitative investigation revealed several potential explanations, including code complexity, developer experience, the type of functionality, and remote development teams. All these factors were related to the level of coverage and quality, with coverage having an effect even after these adjustments. We also find that the test effort increases exponentially with test coverage, but the redu...

• Type: Conference
• Year: 2009
• Where: ESEM
• Authors: Audris Mockus, Nachiappan Nagappan, Trung T. Dinh-Trong
• Keywords: Coverage Reflects Test, ESEM 2009, Software Engineering, Test Coverage, Test Effectiveness