Discriminative separable nonnegative matrix factorization by structured sparse regularization

Non-negative matrix factorization (NMF) is one of the most important models for learning compact representations of high-dimensional data. With the separability condition, separable NMF further enjoys a global optimal solution. However, separable NMF is unable to make use of data label information and thus unfavourable for supervised learning problems. In this paper, we propose discriminative separable NMF (DS-NMF), which extends separable NMF by encoding data label information into data representations. Assuming that each conical basis vector under the separability condition is only contributable to representing data from a few classes, DS-NMF exploits a structured sparse regularization to learning a sparse data representation and provides higher discrimination power than the standard separable NMF. Empirical evaluations on face recognition and scene classification problems confirm the effectiveness of DS-NMF and its superiority to separable NMF. HighlightsPropose a discriminative separable non-negative matrix factorisation (DSNMF) model.Derive an efficient first-order algorithm to learn DS-NMF.Apply DS-NMF to face and scene image classification.

[1]  J KriegmanDavid,et al.  Eigenfaces vs. Fisherfaces , 1997 .

[2]  Joel A. Tropp,et al.  Factoring nonnegative matrices with linear programs , 2012, NIPS.

[3]  Nicolas Gillis,et al.  Semidefinite Programming Based Preconditioning for More Robust Near-Separable Nonnegative Matrix Factorization , 2013, SIAM J. Optim..

[4]  Sanjeev Arora,et al.  Computing a nonnegative matrix factorization -- provably , 2011, STOC '12.

[5]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[6]  Haesun Park,et al.  Toward Faster Nonnegative Matrix Factorization: A New Algorithm and Comparisons , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[7]  Nicolas Gillis,et al.  Fast and Robust Recursive Algorithmsfor Separable Nonnegative Matrix Factorization , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  TorralbaAntonio,et al.  Modeling the Shape of the Scene , 2001 .

[9]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[10]  Inderjit S. Dhillon,et al.  Fast coordinate descent methods with variable selection for non-negative matrix factorization , 2011, KDD.

[11]  John Shawe-Taylor,et al.  MahNMF: Manhattan Non-negative Matrix Factorization , 2012, ArXiv.

[12]  D. B. Gerham.  Characterizing virtual eigensignatures for general purpose face recognition , 1998 .

[13]  P. Gács,et al.  Algorithms , 1992 .

[14]  Victoria Stodden,et al.  When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts? , 2003, NIPS.

[15]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[16]  D. B. Graham,et al.  Characterising Virtual Eigensignatures for General Purpose Face Recognition , 1998 .

[17]  Stephen A. Vavasis,et al.  On the Complexity of Nonnegative Matrix Factorization , 2007, SIAM J. Optim..

[18]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[19]  Amy Nicole Langville,et al.  Algorithms, Initializations, and Convergence for the Nonnegative Matrix Factorization , 2014, ArXiv.

[20]  Zhigang Luo,et al.  NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization , 2012, IEEE Transactions on Signal Processing.

[21]  Zan Gao,et al.  Multi-view discriminative and structured dictionary learning with group sparsity for human action recognition , 2015, Signal Process..

[22]  Andy Harter,et al.  Parameterisation of a stochastic model for human face identification , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[23]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Ivan W. Selesnick,et al.  Translation-Invariant Shrinkage/Thresholding of Group Sparse Signals , 2013, ArXiv.

[26]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[27]  Chih-Jen Lin,et al.  Projected Gradient Methods for Nonnegative Matrix Factorization , 2007, Neural Computation.

[28]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[29]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[30]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[31]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[32]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[33]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[34]  Nicolas Gillis,et al.  Robust near-separable nonnegative matrix factorization using linear optimization , 2013, J. Mach. Learn. Res..

[35]  David Mary,et al.  Blind and Fully Constrained Unmixing of Hyperspectral Images , 2014, IEEE Transactions on Image Processing.

[36]  Dacheng Tao,et al.  Divide-and-Conquer Anchoring for Near-Separable Nonnegative Matrix Factorization and Completion in High Dimensions , 2013, 2013 IEEE 13th International Conference on Data Mining.

[37]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.