Smooth nonnegative matrix and tensor factorizations for robust multi-way data analysis

In this paper, we discuss new efficient algorithms for nonnegative matrix factorization (NMF) with smoothness constraints imposed on nonnegative components or factors. Such constraints allow us to alleviate certain ambiguity problems, which facilitates better physical interpretation or meaning. In our approach, various basis functions are exploited to flexibly and efficiently represent the smooth nonnegative components. For noisy input data, the proposed algorithms are more robust than the existing smooth and sparse NMF algorithms. Moreover, we extend the proposed approach to the smooth nonnegative Tucker decomposition and smooth nonnegative canonical polyadic decomposition (also called smooth nonnegative tensor factorization). Finally, we conduct extensive experiments on synthetic and real-world multi-way array data to demonstrate the advantages of the proposed algorithms. HighlightsWe proposed a new efficient algorithm for smooth NMF.Moreover, we extended it to nonnegative tensor factorization (NTF) models.Proposed methods were applied to blind source separation and parts analysis.Proposed smooth NMF/NTF was more robust to noise than existing NMF/NTF.

[1]  Chris H. Q. Ding,et al.  Convex and Semi-Nonnegative Matrix Factorizations , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Andrzej Cichocki,et al.  Blind Image Separation Using Nonnegative Matrix Factorization with Gibbs Smoothing , 2007, ICONIP.

[3]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[4]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[5]  Haiqing Yin,et al.  Bregman iteration algorithm for sparse nonnegative matrix factorizations via alternating l1-norm minimization , 2011, Multidimensional Systems and Signal Processing.

[6]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[7]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[8]  Andrzej Cichocki,et al.  Fast and Efficient Algorithms for Nonnegative Tucker Decomposition , 2008, ISNN.

[9]  Hairong Qi,et al.  Endmember Extraction From Highly Mixed Data Using Minimum Volume Constrained Nonnegative Matrix Factorization , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Rasmus Bro,et al.  MULTI-WAY ANALYSIS IN THE FOOD INDUSTRY Models, Algorithms & Applications , 1998 .

[11]  Andrzej Cichocki,et al.  Constrained non-Negative Matrix Factorization Method for EEG Analysis in Early Detection of Alzheimer Disease , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[12]  Rafal Zdunek,et al.  Approximation of Feature Vectors in Nonnegative Matrix Factorization with Gaussian Radial Basis Functions , 2012, ICONIP.

[13]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[14]  Andrzej Cichocki,et al.  Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations , 2009, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[15]  Slim Essid,et al.  Smooth Nonnegative Matrix Factorization for Unsupervised Audiovisual Document Structuring , 2013, IEEE Transactions on Multimedia.

[16]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[17]  Lu Li,et al.  Smooth incomplete matrix factorization and its applications in image/video denoising , 2013, Neurocomputing.

[18]  Andrzej Cichocki,et al.  Regularized Alternating Least Squares Algorithms for Non-negative Matrix/Tensor Factorization , 2007, ISNN.

[19]  Dietrich Lehmann,et al.  Nonsmooth nonnegative matrix factorization (nsNMF) , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Andrzej Cichocki,et al.  Hierarchical ALS Algorithms for Nonnegative Matrix and 3D Tensor Factorization , 2007, ICA.

[21]  Nicolas Gillis,et al.  Using underapproximations for sparse nonnegative matrix factorization , 2009, Pattern Recognit..

[22]  David J. Kriegman,et al.  Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection , 1996, ECCV.

[23]  J KriegmanDavid,et al.  Eigenfaces vs. Fisherfaces , 1997 .

[24]  Andrzej Cichocki,et al.  Analysis of financial data using non-negative matrix factorisation , 2008 .

[25]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[26]  M. V. Van Benthem,et al.  Fast algorithm for the solution of large‐scale non‐negativity‐constrained least squares problems , 2004 .

[27]  Andrzej Cichocki,et al.  Adaptive blind signal and image processing , 2002 .

[28]  C. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[29]  Jieping Ye,et al.  Tensor Completion for Estimating Missing Values in Visual Data , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[31]  Andrzej Cichocki,et al.  Extended HALS algorithm for nonnegative Tucker decomposition and its applications for multiway analysis and classification , 2011, Neurocomputing.

[32]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[33]  Andrzej Cichocki,et al.  Local Learning Rules for Nonnegative Tucker Decomposition , 2009, ICONIP.

[34]  M. Genton,et al.  Functional Boxplots , 2011 .

[35]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[36]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[37]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[38]  Takio Kurita,et al.  Automatic factorization of biological signals by using Boltzmann non-negative matrix factorization , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[39]  Haesun Park,et al.  Sparse Nonnegative Matrix Factorization for Clustering , 2008 .

[40]  Daniel D. Lee,et al.  Multiplicative Updates for Nonnegative Quadratic Programming , 2007, Neural Computation.

[41]  Seungjin Choi,et al.  Nonnegative Tucker Decomposition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Patrik O. Hoyer,et al.  Non-negative Matrix Factorization with Sparseness Constraints , 2004, J. Mach. Learn. Res..

[43]  A. Cichocki,et al.  Tensor decompositions for feature extraction and classification of high dimensional datasets , 2010 .