Geometrical Properties of Nu Support Vector Machines with Different Norms

By employing the L1 or L norms in maximizing margins, support vector machines (SVMs) result in a linear programming problem that requires a lower computational load compared to SVMs with the L2 norm. However, how the change of norm affects the generalization ability of SVMs has not been clarified so far except for numerical experiments. In this letter, the geometrical meaning of SVMs with the Lp norm is investigated, and the SVM solutions are shown to have rather little dependency on p.

[1]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[2]  Kazushi Ikeda,et al.  Geometry of Admissible Parameter Region in Neural Learning , 1996 .

[3]  Kazushi Ikeda,et al.  Geometry and learning curves of kernel methods with polynomial kernels , 2004, Systems and Computers in Japan.

[4]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[5]  Ralf Herbrich,et al.  Learning Kernel Classifiers: Theory and Algorithms , 2001 .

[7]  Kazushi Ikeda,et al.  Effects of Soft Margins on Learning Curves of Support Vector Machines , 2004 .

[8]  Jinbo Bi,et al.  A geometric approach to support vector regression , 2003, Neurocomputing.

[9]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[10]  Olvi L. Mangasarian,et al.  Arbitrary-norm separating plane , 1999, Oper. Res. Lett..

[11]  Alexander J. Smola,et al.  Advances in Large Margin Classifiers , 2000 .

[12]  Kazushi Ikeda,et al.  Effects of kernel function on Nu support vector machines in extreme cases , 2006, IEEE Transactions on Neural Networks.

[13]  Kristin P. Bennett,et al.  Duality and Geometry in SVM Classifiers , 2000, ICML.

[14]  Kazushi Ikeda.  An Asymptotic Statistical Theory of Polynomial Kernel Methods , 2004, Neural Computation.

[15]  Noboru Murata,et al.  Support vector machines with different norms: motivation, formulations and results , 2001, Pattern Recognit. Lett..

[16]  Kazushi Ikeda,et al.  An asymptotic statistical analysis of support vector machines with soft margins , 2005, Neural Networks.

[17]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[18]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[19]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.