Sparse LSSVM in Primal Using Cholesky Factorization for Large-Scale Problems

Abstract—For SVM learning, LSSVM, derived by duality (DLSSVM), is a widely used model, because it has an explicit solution. One obvious limitation of the model is that the solution lacks sparseness, which limits it from training large-scale problems efficiently. In this paper, we derive an equivalent LSSVM model in primal space (P-LSSVM) by the representer theorem and prove that P-LSSVM can be solved exactly at some sparse solutions for problems with low-rank kernel matrices. Two algorithms are proposed for finding the sparse (approximate) solution of P-LSSVM by Cholesky factorization. One is based on the decomposition of the kernel matrix K as P P with the best low rank matrix P approximately by pivoting Cholesky factorization. The other is based on solving P-LSSVM by approximating the Cholesky factorization of the Hessian matrix with rankone update scheme. For linear learning problems, theoretical analysis and experimental results support that P-LSSVM can give the sparsest soluti...

• Type: Journal
• Year: 2016
• Where: TNN
• Authors: Shuisheng Zhou
• Keywords: TNN 2016