Stable Orthogonal Local Discriminant Embedding for Linear Dimensionality Reduction

Manifold learning is widely used in machine learning and pattern recognition. However, manifold learning only considers the similarity of samples belonging to the same class and ignores the within-class variation of data, which will impair the generalization and stableness of the algorithms. For this purpose, we construct an adjacency graph to model the intraclass variation that characterizes the most important properties, such as diversity of patterns, and then incorporate the diversity into the discriminant objective function for linear dimensionality reduction. Finally, we introduce the orthogonal constraint for the basis vectors and propose an orthogonal algorithm called stable orthogonal local discriminate embedding. Experimental results on several standard image databases demonstrate the effectiveness of the proposed dimensionality reduction approach.

[1]  Kun Zhou,et al.  Locality Sensitive Discriminant Analysis , 2007, IJCAI.

[2]  Nazli Ikizler-Cinbis,et al.  Object, Scene and Actions: Combining Multiple Features for Human Action Recognition , 2010, ECCV.

[3]  Yousef Saad,et al.  Orthogonal Neighborhood Preserving Projections: A Projection-Based Dimensionality Reduction Technique , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jiawei Han,et al.  Isometric Projection , 2007, AAAI.

[5]  Wei Wu,et al.  Fusion of Multiple Features and Supervised Learning for Chinese OOV Term Detection and POS Guessing , 2011, IJCAI.

[6]  Daoqiang Zhang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2006, IEEE Transactions on Neural Networks.

[7]  Dong Xu,et al.  Regularized Trace Ratio Discriminant Analysis with Patch Distribution Feature for human gait recognition , 2010, 2010 IEEE International Conference on Image Processing.

[8]  Yi Wu,et al.  Stable local dimensionality reduction approaches , 2009, Pattern Recognit..

[9]  Stephen Lin,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[11]  Feiping Nie,et al.  Cauchy Graph Embedding , 2011, ICML.

[12]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[13]  Hujun Bao,et al.  Laplacian Regularized Gaussian Mixture Model for Data Clustering , 2011, IEEE Transactions on Knowledge and Data Engineering.

[14]  Jieping Ye,et al.  Null space versus orthogonal linear discriminant analysis , 2006, ICML '06.

[15]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[16]  Feiping Nie,et al.  Trace Ratio Problem Revisited , 2009, IEEE Transactions on Neural Networks.

[17]  Zhigang Luo,et al.  Non-Negative Patch Alignment Framework , 2011, IEEE Transactions on Neural Networks.

[18]  Xuelong Li,et al.  Discriminative Orthogonal Neighborhood-Preserving Projections for Classification , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[19]  Chengjun Liu,et al.  Gabor feature based classification using the enhanced fisher linear discriminant model for face recognition , 2002, IEEE Trans. Image Process..

[20]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[21]  Junbin Gao,et al.  Comprehensive Analysis for the Local Fisher Discriminant Analysis , 2009, Int. J. Pattern Recognit. Artif. Intell..

[22]  Hui Xu,et al.  Two-dimensional supervised local similarity and diversity projection , 2010, Pattern Recognit..

[23]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[24]  Feiping Nie,et al.  Learning an Orthogonal and Smooth Subspace for Image Classification , 2009, IEEE Signal Processing Letters.

[25]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[26]  De-Shuang Huang,et al.  Locally linear discriminant embedding: An efficient method for face recognition , 2008, Pattern Recognit..

[27]  Ivor W. Tsang,et al.  Flexible Manifold Embedding: A Framework for Semi-Supervised and Unsupervised Dimension Reduction , 2010, IEEE Transactions on Image Processing.

[28]  Xiaogang Wang,et al.  Using random subspace to combine multiple features for face recognition , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[29]  Xuelong Li,et al.  Non-negative graph embedding , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[31]  Dong Xu,et al.  Patch Distribution Compatible Semisupervised Dimension Reduction for Face and Human Gait Recognition , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[32]  Hwann-Tzong Chen,et al.  Local discriminant embedding and its variants , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[33]  Feiping Nie,et al.  Optimal Dimensionality Discriminant Analysis and Its Application to Image Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Stephen P. Boyd,et al.  The Fastest Mixing Markov Process on a Graph and a Connection to a Maximum Variance Unfolding Problem , 2006, SIAM Rev..

[35]  Chun Chen,et al.  Graph Regularized Sparse Coding for Image Representation , 2011, IEEE Transactions on Image Processing.

[36]  Masashi Sugiyama,et al.  Dimensionality Reduction of Multimodal Labeled Data by Local Fisher Discriminant Analysis , 2007, J. Mach. Learn. Res..

[37]  Stephen Lin,et al.  Marginal Fisher Analysis and Its Variants for Human Gait Recognition and Content- Based Image Retrieval , 2007, IEEE Transactions on Image Processing.

[38]  Jingjing Liu,et al.  Enhanced fisher discriminant criterion for image recognition , 2012, Pattern Recognit..

[39]  Jiawei Han,et al.  Orthogonal Laplacianfaces for Face Recognition , 2006, IEEE Transactions on Image Processing.

[40]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Feiping Nie,et al.  Neighborhood MinMax Projections , 2007, IJCAI.

[42]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[43]  Haixian Wang,et al.  Locality-Preserved Maximum Information Projection , 2008, IEEE Transactions on Neural Networks.

[44]  Xiaojun Wu,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  L. Duchene,et al.  An Optimal Transformation for Discriminant and Principal Component Analysis , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[46]  Zhigang Luo,et al.  Manifold Regularized Discriminative Nonnegative Matrix Factorization With Fast Gradient Descent , 2011, IEEE Transactions on Image Processing.

[47]  Feiping Nie,et al.  Orthogonal vs. uncorrelated least squares discriminant analysis for feature extraction , 2012, Pattern Recognit. Lett..

[48]  Kilian Q. Weinberger,et al.  An Introduction to Nonlinear Dimensionality Reduction by Maximum Variance Unfolding , 2006, AAAI.

[49]  Jiawei Han,et al.  Non-negative Matrix Factorization on Manifold , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[50]  Jiawei Han,et al.  Learning a Maximum Margin Subspace for Image Retrieval , 2008, IEEE Transactions on Knowledge and Data Engineering.

[51]  Nicolas Le Roux,et al.  Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering , 2003, NIPS.

[52]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.