What Test Oracle Should I Use for Effective GUI Testing?

Test designers widely believe that the overall effectiveness and cost of software testing depends largely on the type and number of test cases executed on the software. In this paper we show that the test oracle used during testing also contributes significantly to test effectiveness and cost. A test oracle is a mechanism that determines whether a software executed correctly for a test case. We define a test oracle to contain two essential parts: oracle information that represents expected output, and an oracle procedure that compares the oracle information with the actual output. By varying the level of detail of oracle information and changing the oracle procedure, a test designer can create different types of test oracles. We design 11 types of test oracles and empirically compare them on four software systems. We seed faults in each software to create 100 faulty versions, execute 600 test cases on each version, for all 11 types of oracles. In all, we report results of 660,000 te...

• Type: Conference
• Year: 2003
• Where: KBSE
• Authors: Atif M. Memon, Ishan Banerjee, Adithya Nagarajan
• Keywords: KBSE 2003, Oracle Information, Software Engineering, Test Cases, Test Oracles