Linear dimensionality reduction using relevance weighted LDA

The linear discriminant analysis (LDA) is one of the most traditional linear dimensionality reduction methods. This paper incorporates the inter-class relationships as relevance weights into the estimation of the overall within-class scatter matrix in order to improve the performance of the basic LDA method and some of its improved variants. We demonstrate that in some specific situations the standard multi-class LDA almost totally fails to find a discriminative subspace if the proposed relevance weights are not incorporated. In order to estimate the relevance weights of individual within-class scatter matrices, we propose several methods of which one employs the evolution strategies.

[1]  Chengjun Liu,et al.  Evolutionary Pursuit and Its Application to Face Recognition , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Keinosuke Fukunaga,et al.  Introduction to Statistical Pattern Recognition , 1972 .

[3]  Ravi Kothari,et al.  Fractional-Step Dimensionality Reduction , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Hua Yu,et al.  A direct LDA algorithm for high-dimensional data - with application to face recognition , 2001, Pattern Recognit..

[5]  C. R. Rao,et al.  The Utilization of Multiple Measurements in Problems of Biological Classification , 1948 .

[6]  Robert P. W. Duin,et al.  Multiclass Linear Dimension Reduction by Weighted Pairwise Fisher Criteria , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Kalyanmoy Deb,et al.  On self-adaptive features in real-parameter evolutionary algorithms , 2001, IEEE Trans. Evol. Comput..

[8]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[9]  Anil K. Jain,et al.  Dimensionality reduction using genetic algorithms , 2000, IEEE Trans. Evol. Comput..

[10]  Robert P. W. Duin,et al.  Linear dimensionality reduction via a heteroscedastic extension of LDA: the Chernoff criterion , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  LoogMarco,et al.  Linear Dimensionality Reduction via a Heteroscedastic Extension of LDA , 2004 .

[12]  Andreas G. Andreou,et al.  Heteroscedastic discriminant analysis and reduced rank HMMs for improved speech recognition , 1998, Speech Commun..

[13]  Chengjun Liu,et al.  Robust coding schemes for indexing and retrieval from large face databases , 2000, IEEE Trans. Image Process..

[14]  R. Fisher.  THE STATISTICAL UTILIZATION OF MULTIPLE MEASUREMENTS , 1938 .

[15]  Jing-Yu Yang,et al.  Face recognition based on the uncorrelated discriminant transformation , 2001, Pattern Recognit..

[16]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[17]  Xin Yao,et al.  Fast Evolution Strategies , 1997, Evolutionary Programming.

[18]  R. Tibshirani,et al.  Discriminant Analysis by Gaussian Mixtures , 1996 .

[19]  G AndreouAndreas,et al.  Heteroscedastic discriminant analysis and reduced rank HMMs for improved speech recognition , 1998 .

[20]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[21]  Stavros J. Perantonis,et al.  On the relation between discriminant analysis and mutual information for supervised linear feature extraction , 2004, Pattern Recognit..

[22]  Manabu Kotani,et al.  Feature extraction using evolutionary computation , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[23]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[24]  Louis A. Tamburino,et al.  Evolving pattern recognition systems , 2002, IEEE Trans. Evol. Comput..

[25]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[26]  Daijin Kim,et al.  A Handwritten Numeral Character Classification Using Tolerant Rough Set , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[28]  Ponnuthurai N. Suganthan,et al.  Structural pattern recognition using genetic algorithms with specialized operators , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[29]  M. Ahmadi,et al.  Automatic localization of craniofacial landmarks for assisted cephalometry , 2003, Proceedings of the 2003 International Symposium on Circuits and Systems, 2003. ISCAS '03..

[30]  Xin Yao,et al.  A constructive algorithm for training cooperative neural network ensembles , 2003, IEEE Trans. Neural Networks.

[31]  David A. Landgrebe,et al.  Covariance Matrix Estimation and Classification With Limited Training Data , 1996, IEEE Trans. Pattern Anal. Mach. Intell..