The reliability of confidence intervals for computational effort comparisons

This paper analyses the reliability of confidence intervals for Koza's computational effort statistic. First, we conclude that dependence between the observed minimum generation and the observed cumulative probability of success leads to the production of more reliable confidence intervals for our preferred method. Second, we show that confidence intervals from 80% to 95% have appropriate levels of performance. Third, simulated data is used to consider the effect of large minimum generations and the confidence intervals are again found to be reliable. Finally, results from four large datasets collected from real genetic programming experiments are used to provide even more empirical evidence that the method for producing confidence intervals is reliable. Categories and Subject Descriptors I.2.2 [Artificial Intelligence]: Automatic Programming-Program synthesis General Terms Experimentation, Measurement, Performance Keywords Genetic Programming, Computational Effort, Confidence In...

• Type: Conference
• Year: 2007
• Where: GECCO
• Authors: Matthew Walker, Howard Edwards, Chris H. Messom
• Keywords: Confidence Intervals, GECCO 2007, Minimum Generations, Optimization, Reliable Confidence Intervals