Efficient Large Scale Linear Programming Support Vector Machines

This paper presents a decomposition method for efficiently constructing 1 -norm Support Vector Machines (SVMs). The decomposition algorithm introduced in this paper possesses many desirable properties. For example, it is provably convergent, scales well to large datasets, is easy to implement, and can be extended to handle support vector regression and other SVM variants. We demonstrate the efficiency of our algorithm by training on (dense) synthetic datasets of sizes up to 20 million points (in R 32 ). The results show our algorithm to be several orders of magnitude faster than a previously published method for the same task. We also present experimental results on real data sets-our method is seen to be not only very fast, but also highly competitive against the leading SVM implementations.

[1]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[2]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[3]  Robert Tibshirani,et al.  1-norm Support Vector Machines , 2003, NIPS.

[4]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[5]  Clifford Hildreth,et al.  A quadratic programming procedure , 1957 .

[6]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[7]  O. Mangasarian.  Normal solutions of linear programs , 1984 .

[8]  David R. Musicant,et al.  Active Support Vector Machine Classification , 2000, NIPS.

[9]  O. Mangasarian,et al.  Massive data discrimination via linear support vector machines , 2000 .

[10]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[11]  Olvi L. Mangasarian,et al.  Exact 1-Norm Support Vector Machines Via Unconstrained Convex Differentiable Minimization , 2006, J. Mach. Learn. Res..

[12]  Andrzej Stachurski,et al.  Parallel Optimization: Theory, Algorithms and Applications , 2000, Scalable Comput. Pract. Exp..

[13]  R. C. Williamson,et al.  Classification on proximity data with LP-machines , 1999 .

[14]  Yiming Yang,et al.  RCV1: A New Benchmark Collection for Text Categorization Research , 2004, J. Mach. Learn. Res..

[15]  Kristin P. Bennett,et al.  Combining support vector and mathematical programming methods for classification , 1999 .

[16]  Olivier Chapelle,et al.  Training a Support Vector Machine in the Primal , 2007, Neural Computation.