Assessing Oracle Quality with Checked Coverage

—A known problem of traditional coverage metrics is that they do not assess oracle quality—that is, whether the computation result is actually checked against expectations. In this paper, we introduce the concept of checked coverage—the dynamic slice of covered statements that actually influence an oracle. Our experiments on seven open-source projects show that checked coverage is a sure indicator for oracle quality— and even more sensitive than mutation testing, its much more demanding alternative. Keywords-test suite quality, coverage metrics, dynamic slicing, mutation testing

• Type: Journal
• Year: 2011
• Where: ICST
• Authors: David Schuler, Andreas Zeller
• Keywords: Coverage—the Dynamic Slice, ICST 2011, Mutation Testing, Software Engineering, —A Known Problem