Boosting strategy for classification

This paper introduces a strategy for training ensemble classifiers by analysing boosting within margin theory. We present a bound on the generalisation error of ensembled classifiers in terms of the 2-norm of the margin slack vector. We develop an effective, adaptive and robust boosting algorithm, DMBoost, by optimising this bound. The soft margin based quadratic loss function is insensitive to points having a large margin. The algorithm improves the generalisation performance of a system by ignoring the examples having small or negative margin. We evaluate the efficacy of the proposed method by applying it to a text categorization task. Experimental results show that DMBoost performs significantly better than AdaBoost, hence validating the effectiveness of the method. Furthermore, experimental results on UCI data sets demonstrate that DMBoost generally outperforms AdaBoost.

• Type: Journal
• Year: 2002
• Where: IDA
• Authors: Huma Lodhi, Grigoris J. Karakoulas, John Shawe-Taylor
• Keywords: Generalisation Error, IDA 2002, Information Technology, Margin Theory, Soft Margin