Boosting Classifiers with Tightened L0-Relaxation Penalties

We propose a novel boosting algorithm which improves on current algorithms for weighted voting classification by striking a better balance between classification accuracy and the sparsity of the weight vector. In order to justify our optimization formulations, we first consider a novel integer linear program as a model for sparse classifier selection, generalizing the minimum disagreement halfspace problem whose complexity has been investigated in computational learning theory. Specifically, our mixed integer problem is that of finding a separating hyperplane with minimum empirical error subject to an L0-norm penalty. We note that common "soft margin" linear programming formulations for robust classification are equivalent to the continuous relaxation of our formulation. Since the initial continuous relaxation is weak, we suggest a tighter relaxation, using novel cutting planes, to better approximate the integer solution. To solve this relaxation, we propose a new boosting a...

• Type: Conference
• Year: 2010
• Where: ICML
• Authors: Noam Goldberg, Jonathan Eckstein
• Keywords: Boosting Algorithm, Continuous Relaxation, ICML 2010, Linear Programming, Machine Learning