Weighted Additive Criterion for Linear Dimension Reduction

Linear discriminant analysis (LDA) for dimension reduction has been applied to a wide variety of face recognition tasks. However, it has two major problems. First, it suffers from the small sample size problem when dimensionality is greater than the sample size. Second, it creates subspaces that favor well separated classes over those that are not. In this paper, we propose a simple weighted criterion for linear dimension reduction that addresses the above two problems associated with LDA. In addition, there are well established numerical procedures such as semi-definite programming for efficiently computing the proposed criterion. We demonstrate the efficacy of our proposal and compare it against other competing techniques using a number of examples.

[1]  Jos F. Sturm,et al.  A Matlab toolbox for optimization over symmetric cones , 1999 .

[2]  Jing-Yu Yang,et al.  A generalized optimal set of discriminant vectors , 1992, Pattern Recognit..

[3]  G. Stewart.  Introduction to matrix computations , 1973 .

[4]  Stephen P. Boyd,et al.  Semidefinite Programming , 1996, SIAM Rev..

[5]  Anil K. Jain,et al.  Small Sample Size Effects in Statistical Pattern Recognition: Recommendations for Practitioners , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[7]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[8]  Sarunas Raudys,et al.  On Dimensionality, Sample Size, Classification Error, and Complexity of Classification Algorithm in Pattern Recognition , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[10]  Anil K. Jain,et al.  39 Dimensionality and sample size considerations in pattern recognition practice , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[11]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[12]  Robert P. W. Duin,et al.  Multiclass Linear Dimension Reduction by Weighted Pairwise Fisher Criteria , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[14]  Qi Tian,et al.  Image Classification By The Foley-Sammon Transform , 1986 .

[15]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[16]  Tao Jiang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2003, IEEE Transactions on Neural Networks.

[17]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[18]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[19]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[20]  Hua Yu,et al.  A direct LDA algorithm for high-dimensional data - with application to face recognition , 2001, Pattern Recognit..

[21]  Juyang Weng,et al.  Using Discriminant Eigenfeatures for Image Retrieval , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Hanqing Lu,et al.  Solving the small sample size problem of LDA , 2002, Object recognition supported by user interaction for service robots.

[23]  Ja-Chen Lin,et al.  A new LDA-based face recognition system which can solve the small sample size problem , 1998, Pattern Recognit..

[24]  Gábor Pataki,et al.  On the Rank of Extreme Matrices in Semidefinite Programs and the Multiplicity of Optimal Eigenvalues , 1998, Math. Oper. Res..