Linear regression based projections for dimensionality reduction

Abstract In graph embedding based dimensionality reduction methods, the number of K-nearest neighbors is usually needed to be manually chosen in high dimensional space. Graph construction by different number of K-nearest neighbor changes dramatically and seriously affects the performance of graph embedding based dimensionality reduction. How to automatically construct a graph is very important. In this paper, first, a discriminative L2-graph is investigated. It computes the edge weights using the class-specific samples and weighted ridge regression, avoiding manually choosing the K-nearest neighbors in traditional graph construction. Second, a discriminative L2-graph based dimensionality reduction method is proposed, named Linear Regression based Projections (LRP). LRP minimizes the ratio between the local compactness information and the total separability information to seek the optimal projection matrix. LRP is much faster than its counterparts, Sparsity Preserving Projections (SPP) and Collaborative Representation based Projections (CRP), since LRP is supervised and computes edge weights using class-specific samples while SPP and CRP are unsupervised and compute edge weights using all samples. The experimental results on benchmark face image databases show that the proposed LRP outperforms many existing representative linear dimensionality reduction methods.

[1]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression Database , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Hong Zhao,et al.  An exponent weighted algorithm for minimal cost feature selection , 2016, Int. J. Mach. Learn. Cybern..

[3]  Lei Zhang,et al.  A multi-manifold discriminant analysis method for image feature extraction , 2011, Pattern Recognit..

[4]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[5]  Jian Yang,et al.  Sparse Approximation to the Eigensubspace for Discrimination , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[6]  David J. Kriegman,et al.  Acquiring linear subspaces for face recognition under variable lighting , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[8]  I. Jolliffe Principal Component Analysis , 2002 .

[9]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[10]  Mohammed Bennamoun,et al.  Linear Regression for Face Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  A. Martínez,et al.  The AR face databasae , 1998 .

[12]  Zhenyu Wang,et al.  A collaborative representation based projections method for feature extraction , 2015, Pattern Recognit..

[13]  Hyeonjoon Moon,et al.  The FERET Evaluation Methodology for Face-Recognition Algorithms , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Zizhu Fan,et al.  Weighted sparse representation for face recognition , 2015, Neurocomputing.

[15]  Bin Luo,et al.  2D-LPP: A two-dimensional extension of locality preserving projections , 2007, Neurocomputing.

[16]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[17]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Jian Yang,et al.  Sparse Representation Classifier Steered Discriminative Projection With Applications to Face Recognition , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[19]  Jun Ye,et al.  Simplified neutrosophic harmonic averaging projection-based method for multiple attribute decision-making problems , 2015, International Journal of Machine Learning and Cybernetics.

[20]  Xiaoyang Tan,et al.  Sparsity preserving projections with applications to face recognition , 2010, Pattern Recognit..

[21]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Gene H. Golub,et al.  Matrix computations , 1983 .

[23]  Yuan-Hai Shao,et al.  Multiple recursive projection twin support vector machine for multi-class classification , 2014, International Journal of Machine Learning and Cybernetics.

[24]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[25]  Shuicheng Yan,et al.  Neighborhood preserving embedding , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[26]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[27]  Limei Zhang,et al.  Graph optimization for dimensionality reduction with sparsity constraints , 2012, Pattern Recognit..

[28]  Wenjia Wang,et al.  Determining appropriate approaches for using data in feature selection , 2017, Int. J. Mach. Learn. Cybern..

[29]  Bin Luo,et al.  Similarity Learning of Manifold Data , 2015, IEEE Transactions on Cybernetics.

[30]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[31]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[32]  Latika Singh,et al.  A hybrid feature selection approach based on improved PSO and filter approaches for image steganalysis , 2016, Int. J. Mach. Learn. Cybern..

[33]  Luc Van Gool,et al.  Adaptive and Weighted Collaborative Representations for image classification , 2014, Pattern Recognit. Lett..

[34]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[35]  Ronen Basri,et al.  Lambertian Reflectance and Linear Subspaces , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Shuicheng Yan,et al.  Learning With $\ell ^{1}$-Graph for Image Analysis , 2010, IEEE Transactions on Image Processing.

[37]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  David G. Stork,et al.  Pattern Classification , 1973 .

[39]  Mikhail Belkin,et al.  Laplacian Eigenmaps for Dimensionality Reduction and Data Representation , 2003, Neural Computation.

[40]  Lei Zhang,et al.  Sparse representation or collaborative representation: Which helps face recognition? , 2011, 2011 International Conference on Computer Vision.

[41]  Zhenyu Wang,et al.  Face recognition using adaptive local ternary patterns method , 2016, Neurocomputing.

[42]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[43]  Terence Sim,et al.  The CMU Pose, Illumination, and Expression (PIE) database , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[44]  Ying-Ke Lei,et al.  Face recognition via Weighted Sparse Representation , 2013, J. Vis. Commun. Image Represent..

[45]  David J. Kriegman,et al.  From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose , 2001, IEEE Trans. Pattern Anal. Mach. Intell..