Learning r-of-k Functions by Boosting

We investigate further improvement of boosting in the case that the target concept belongs to the class of r-of-k threshold Boolean functions, which answer “+1” if at least r of k relevant variables are positive, and answer “−1” otherwise. Given m examples of a r-of-k function and literals as base hypotheses, popular boosting algorithms (e.g., AdaBoost) construct a consistent final hypothesis by using O(k2 log m) base hypotheses. While this convergence speed is tight in general, we show that a modification of AdaBoost (confidence-rated AdaBoost [SS99] or InfoBoost [Asl00]) can make use of the property of r-of-k functions that make less error on one-side to find a consistent final hypothesis by using O(kr log m) hypotheses. Our result extends the previous investigation by Hatano and Warmuth [HW04] and gives more general examples where confidence-rated AdaBoost or InfoBoost has an advantage over AdaBoost.

• Type: Conference
• Year: 2004
• Where: ALT
• Authors: Kohei Hatano, Osamu Watanabe
• Keywords: ALT 2004, Base Hypotheses, Consistent final Hypothesis, Confidence-rated Adaboost, Machine Learning