Boosting and Maximum Likelihood for Exponential Models

We derive an equivalence between AdaBoost and the dual of a convex optimization problem, showing that the only difference between minimizing the exponential loss used by AdaBoost and maximum likelihood for exponential models is that the latter requires the model to be normalized to form a conditional probability distribution over labels. In addition to establishing a simple and easily understood connection between the two methods, this framework enables us to derive new regularization procedures for boosting that directly correspond to penalized maximum likelihood. Experiments on UCI datasets support our theoretical analysis and give additional insight into the relationship between boosting and logistic regression.

• Type: Conference
• Year: 2001
• Where: NIPS
• Authors: Guy Lebanon, John D. Lafferty
• Keywords: Convex Optimization Problem, Maximum Likelihood, NIPS 2001, NIPS 2007, Penalized Maximum Likelihood