Comparing Linear Discriminant Analysis and Support Vector Machines

Both Linear Discriminant Analysis and Support Vector Machines compute hyperplanes that are optimal with respect to their individual objectives. However, there can be vast differences in performance between the two techniques depending on the extent to which their respective assumptions agree with problems at hand. In this paper we compare the two techniques analytically and experimentally using a number of data sets. For analytical comparison purposes, a unified representation is developed and a metric of optimality is proposed.

[1]  J. Mercer Functions of positive and negative type, and their connection with the theory of integral equations , 1909 .

[2]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[3]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[4]  G. McLachlan Discriminant Analysis and Statistical Pattern Recognition , 1992 .

[5]  Keinosuke Fukunaga,et al.  Statistical Pattern Recognition , 1993, Handbook of Pattern Recognition and Computer Vision.

[6]  Jerome H. Friedman,et al.  Flexible Metric Nearest Neighbor Classification , 1994 .

[7]  Robert Tibshirani,et al.  Discriminant Adaptive Nearest Neighbor Classification , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  J. C. BurgesChristopher A Tutorial on Support Vector Machines for Pattern Recognition , 1998 .

[9]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[10]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[11]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[12]  B. Schölkopf,et al.  Advances in kernel methods: support vector learning , 1999 .

[13]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[14]  Colin Campbell,et al.  Bayes Point Machines , 2001, J. Mach. Learn. Res..

[15]  Michael I. Jordan,et al.  Kernel independent component analysis , 2003 .