Sphered Support Vector Machine

We introduce an objective function for support vector machines (SVMs) which is scale invariant and leads to improved bounds on the generalization error. Standard SVMs and their margin bounds are not invariant under linear transformation. The new objective function leads to new SVM approach, called “Sphered Support Vector Machine” (S-SVM). The S-SVMs are kernelized and regularized but can be fast implemented by an incremental learning method and do not require kernel PCA like the approaches in (Chapelle and Schölkopf, 2002). On real world benchmark datasets we compare the new S-SVMs to standard ν-SVMs, where the S-SVMs showed comparable to superior classification performance to νSVMs.

[1]  유정완,et al.  Learning with Kernels , 2011 .

[2]  Vanessa Su Lee Goh,et al.  Adaptive and Learning Systems for Signal, Processing, Communications, and Control , 2009 .

[3]  Thore Graepel,et al.  A PAC-Bayesian margin bound for linear classifiers , 2002, IEEE Trans. Inf. Theory.

[4]  Nello Cristianini,et al.  On the generalization of soft margin algorithms , 2002, IEEE Trans. Inf. Theory.

[5]  Christopher K. I. Williams,et al.  Learning With Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2001 .

[6]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[7]  Gunnar Rätsch,et al.  Soft Margins for AdaBoost , 2001, Machine Learning.

[8]  John Shawe-Taylor,et al.  Structural Risk Minimization Over Data-Dependent Hierarchies , 1998, IEEE Trans. Inf. Theory.

[9]  Bernhard Schölkopf,et al.  Prior Knowledge in Support Vector Kernels , 1997, NIPS.

[10]  B. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[11]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[12]  John C. Platt,et al.  Fast Training of Support Vector Machines using Sequential Minimal Optimization , 2000 .

[13]  John Shawe-Taylor,et al.  A framework for structural risk minimisation , 1996, COLT '96.

[14]  H. Luetkepohl The Handbook of Matrices , 1996 .

[15]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..