Multiclass Learning, Boosting, and Error-Correcting Codes

We focus on methods to solve multiclass learning problems by using only simple and efficient binary learners. We investigate the approach of Dietterich and Bakiri [2] based on error-correcting codes (which we call ECC). We distill error correlation as one of the key parameters influencing the performance of the ECC approach, and prove upper and lower bounds on the training error of the final hypothesis in terms of the error-correlation between the various binary hypotheses. Boosting is a powerful and well-studied learning technique that appears to annul error correlation disadvantages by cleverly weighting training examples and hypotheses. An interesting algorithm called ADABOOST.OC [12] combines boosting with the ECC approach and gives an algorithm that has the performance advantages of boosting and at the same time relies only on simple binary weak learners. We propose a variant of this algorithm, which we call ADABOOST.ECC, that, by using a different weighting of the votes of th...

• Type: Conference
• Year: 1999
• Where: COLT
• Authors: Venkatesan Guruswami, Amit Sahai
• Keywords: COLT 1999, ECC Approach, Machine Learning, Multiclass Learning, Multiclass Learning Problems