A Simple Geometric Interpretation of SVM using Stochastic Adversaries

We present a minimax framework for classification that considers stochastic adversarial perturbations to the training data. We show that for binary classification it is equivalent to SVM, but with a very natural interpretation of regularization parameter. In the multiclass case, we obtain that our formulation is equivalent to regularizing the hinge loss with the maximum norm of the weight vector (i.e., the two-infinity norm). We test this new regularization scheme and show that it is competitive with the Frobenius regularization commonly used for multiclass SVM. We proceed to analyze various forms of stochastic perturbations and obtain compact optimization problems for the optimal classifiers. Taken together, our results illustrate the advantage of using stochastic perturbations rather than deterministic ones, as well as offer a simple geometric interpretation for SVM optimization.

• Type: Journal
• Year: 2012
• Where: JMLR
• Authors: Roi Livni, Koby Crammer, Amir Globerson
• Keywords: JMLR 2012, Maximum Norm, Programming Languages, Stochastic Perturbations, Weight Vector