Direct convex relaxations of sparse SVM

Although support vector machines (SVMs) for binary classification give rise to a decision rule that only relies on a subset of the training data points (support vectors), it will in general be based on all available features in the input space. We propose two direct, novel convex relaxations of a nonconvex sparse SVM formulation that explicitly constrains the cardinality of the vector of feature weights. One relaxation results in a quadratically-constrained quadratic program (QCQP), while the second is based on a semidefinite programming (SDP) relaxation. The QCQP formulation can be interpreted as applying an adaptive soft-threshold on the SVM hyperplane, while the SDP formulation learns a weighted inner-product (i.e. a kernel) that results in a sparse hyperplane. Experimental results show an increase in sparsity while conserving the generalization performance compared to a standard as well as a linear programming SVM.

• Type: Conference
• Year: 2007
• Where: ICML
• Authors: Antoni B. Chan, Nuno Vasconcelos, Gert R. G. Lanckriet
• Keywords: ICML 2007, Linear Programming Svm, Machine Learning, Sparse Svm Formulation, SVM Hyperplane