Predicting good probabilities with supervised learning

We examine the relationship between the predictions made by different learning algorithms and true posterior probabilities. We show that maximum margin methods such as boosted trees and boosted stumps push probability mass away from 0 and 1 yielding a characteristic sigmoid shaped distortion in the predicted probabilities. Models such as Naive Bayes, which make unrealistic independence assumptions, push probabilities

• Type: Conference
• Year: 2005
• Where: ICML
• Authors: Alexandru Niculescu-Mizil, Rich Caruana
• Keywords: ICML 2005, Machine Learning, Stumps Push Probability, True Posterior Probabilities, Unrealistic Independence Assumptions