An Empirical Study on Testing and Fault Tolerance for Software Reliability Engineering

Software testing and software fault tolerance are two major techniques for developing reliable software systems, yet limited empirical data are available in the literature to evaluate their effectiveness. We conducted a major experiment to engage 34 programming teams to independently develop multiple software versions for an industry-scale critical flight application, and collected faults detected in these program versions. To evaluate the effectiveness of software testing and software fault tolerance, mutants were created by injecting real faults occurred in the development stage. The nature, manifestation, detection, and correlation of these faults were carefully investigated. The results show that coverage testing is generally an effective mean to detecting software faults, but the effectiveness of testing coverage is not equivalent to that of mutation coverage, which is a more truthful indicator of testing quality. We also found that exact faults found among versions are very limi...

• Type: Conference
• Year: 2003
• Where: ISSRE
• Authors: Michael R. Lyu, Zubin Huang, Sam K. S. Sze, Xia Cai
• Keywords: Faults, ISSRE 2003, Software Fault Tolerance, Software Testing