Exact Mean Absolute Error of Baseline Predictor, MARP0

Shepperd and MacDonell “Evaluating prediction systems in software project estimation”. Information and Software Technology 54 (8), 820–827, 2012, proposed an improved measure of the effectiveness of predictors based on comparing them with random guessing. They suggest estimating the performance of random guessing using a Monte Carlo scheme which unfortunately excludes some correct guesses. This biases their MARP0 to be slightly too big, which in turn causes their standardised accuracy measure SA to over estimate slightly. In commonly used software engineering datasets it is practical to calculate an unbiased MARP0 exactly.

• Type: Journal
• Year: 2016
• Where: INFSOF
• Authors: William B. Langdon, José Javier Dolado, Federica Sarro, Mark Harman
• Keywords: INFSOF 2016, Software Engineering