Bayesian Methods for Support Vector Machines: Evidence and Predictive Class Probabilities

I describe a framework for interpreting Support Vector Machines (SVMs) as maximum a posteriori (MAP) solutions to inference problems with Gaussian Process priors. This probabilistic interpretation can provide intuitive guidelines for choosing a `good' SVM kernel. Beyond this, it allows Bayesian methods to be used for tackling two of the outstanding challenges in SVM classification: how to tune hyperparameters--the misclassification penalty C, and any parameters specifying the kernel--and how to obtain predictive class probabilities rather than the conventional deterministic class label predictions. Hyperparameters can be set by maximizing the evidence; I explain how the latter can be defined and properly normalized. Both analytical approximations and numerical methods (Monte Carlo chaining) for estimating the evidence are discussed. I also compare different methods of estimating class probabilities, ranging from simple evaluation at the MAP or at the posterior average to full aver...

• Type: Journal
• Year: 2002
• Where: ML
• Authors: Peter Sollich
• Keywords: Class Probabilities, Gaussian Process Priors, Machine Learning, ML 2002, Predictive Class Probabilities