Image Classification with Nonnegative Matrix Factorization Based on Spectral Projected Gradient

Nonnegative Matrix Factorization (NMF) is a key tool for model dimensionality reduction in supervised classification. Several NMF algorithms have been developed for this purpose. In a majority of them, the training process is improved by using discriminant or nearest-neighbor graph-based constraints that are obtained from the knowledge on class labels of training samples. The constraints are usually incorporated to NMF algorithms by l 2-weighted penalty terms that involve formulating a large-size weighting matrix. Using the Newton method for updating the latent factors, the optimization problems in NMF become large-scale. However, the computational problem can be considerably alleviated if the modified Spectral Projected Gradient (SPG) that belongs to a class of quasi-Newton methods is used. The simulation results presented for the selected classification problems demonstrate the high efficiency of the proposed method.

[1]  Hans-Paul Schwefel,et al.  Advances in Computational Intelligence , 2003, Natural Computing Series.

[2]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[3]  Shuicheng Yan,et al.  Multiplicative nonnegative graph embedding , 2009, CVPR.

[4]  Mikhail Belkin,et al.  Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering , 2001, NIPS.

[5]  Jacek M. Zurada,et al.  Artificial Intelligence and Soft Computing, 10th International Conference, ICAISC 2010, Zakopane, Poland, June 13-17, 2010, Part I , 2010, International Conference on Artificial Intelligence and Soft Computing.

[6]  Dietrich Lehmann,et al.  Nonsmooth nonnegative matrix factorization (nsNMF) , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Václav Hlavác,et al.  Sequential Coordinate-Wise Algorithm for the Non-negative Least Squares Problem , 2005, CAIP.

[8]  Constantine Kotropoulos,et al.  Musical Instrument Classification using Non-Negative Matrix Factorization Algorithms and Subset Feature Selection , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[9]  Zhigang Luo,et al.  Manifold Regularized Discriminative Nonnegative Matrix Factorization With Fast Gradient Descent , 2011, IEEE Transactions on Image Processing.

[10]  Rafal Zdunek.  Initialization of Nonnegative Matrix Factorization with Vertices of Convex Polytope , 2012, ICAISC.

[11]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[12]  Andrzej Cichocki,et al.  GNMF with Newton-Based Methods , 2013, ICANN.

[13]  Hyunsoo Kim,et al.  Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method , 2008, SIAM J. Matrix Anal. Appl..

[14]  Jiawei Han,et al.  Non-negative Matrix Factorization on Manifold , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[15]  Andrew Zisserman,et al.  Advances in Neural Information Processing Systems (NIPS) , 2007 .

[16]  Rafal Zdunek,et al.  Regularized Active Set Least Squares Algorithm for Nonnegative Matrix Factorization in Application to Raman Spectra Separation , 2011, IWANN.

[17]  Günther Palm,et al.  Artificial Neural Networks and Machine Learning – ICANN 2013 , 2013, Lecture Notes in Computer Science.

[18]  Zhigang Luo,et al.  NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization , 2012, IEEE Transactions on Signal Processing.

[19]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[20]  Andrzej Cichocki,et al.  Nonnegative matrix factorization with constrained second-order optimization , 2007, Signal Process..

[21]  Hai Jin,et al.  Projective Nonnegative Graph Embedding , 2010, IEEE Transactions on Image Processing.

[22]  Chih-Jen Lin,et al.  On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization , 2007, IEEE Transactions on Neural Networks.

[23]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[24]  Xiaofei He,et al.  Locality Preserving Projections , 2003, NIPS.

[25]  Andrzej Cichocki,et al.  Extended HALS algorithm for nonnegative Tucker decomposition and its applications for multiway analysis and classification , 2011, Neurocomputing.

[26]  Stan Z. Li,et al.  Learning spatially localized, parts-based representation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[27]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[28]  Patrik O. Hoyer,et al.  Non-negative sparse coding , 2002, Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing.

[29]  Ioannis Pitas,et al.  Discriminant Non-negative Matrix Factorization and Projected Gradients for Frontal Face Verification , 2008, BIOID.

[30]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[31]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[32]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[33]  Andrzej Cichocki,et al.  Sequential Coordinate-Wise DNMF for Face Recognition , 2010, ICAISC.

[34]  Massimo Tistarelli,et al.  Biometrics and Identity Management , 2008, Lecture Notes in Computer Science.

[35]  Wen Gao,et al.  Unsupervised texture classification: Automatically discover and classify texture patterns , 2008, Image Vis. Comput..

[36]  Nikos D. Sidiropoulos,et al.  Non-Negative Matrix Factorization Revisited: Uniqueness and Algorithm for Symmetric Decomposition , 2014, IEEE Transactions on Signal Processing.

[37]  Yunde Jia,et al.  FISHER NON-NEGATIVE MATRIX FACTORIZATION FOR LEARNING LOCAL FEATURES , 2004 .

[38]  Xuelong Li,et al.  Non-negative graph embedding , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  A. Cichocki,et al.  Tensor decompositions for feature extraction and classification of high dimensional datasets , 2010 .

[40]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[41]  Anastasios Tefas,et al.  Exploiting discriminant information in nonnegative matrix factorization with application to frontal face verification , 2006, IEEE Transactions on Neural Networks.

[42]  Cédric Richard,et al.  Nonnegative matrix factorization with regularization and sparsity-enforcing terms , 2011, 2011 4th IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[43]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .