Multiple Comparisons in Induction Algorithms

Abstract. A single mechanism is responsible for three pathologies of induction algorithms: attribute selection errors, overfitting, and oversearching. In each pathology, induction algorithms compare multiple items based on scores from an evaluation function and select the item with the maximum score. We call this a multiple comparison procedure (MCP ). We analyze the statistical properties of MCP s and show how failure to adjust for these properties leads to the pathologies. We also discuss approaches that can control pathological behavior, including Bonferroni adjustment, randomization testing, and cross-validation.

• Type: Journal
• Year: 2000
• Where: ML
• Authors: David D. Jensen, Paul R. Cohen
• Keywords: Induction Algorithms, Machine Learning, ML 2000, Multiple Comparison Procedure, Single Mechanism