Damped Newton Iterations for Nonnegative Matrix Factorization

Nonnegative Matrix Factorization (NMF) is a robust method for multivariate data analysis, especially as the data is sparse and highly redundant. However, when the redundancy is not very strong, the data is noisy, and no special constraints on the factors to be estimated can be imposed, the factorization is very challenging, even for a small-scale data. To tackle such problems, we propose to use damped Newton iterations. Initially, we update the solution basically with the information on a gradient direction, and gradually steer towards Newton iterations when we observe a decrease in the cost function. We demonstrated experimentally that our approach is superior to many well-known methods for NMF.

[1]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[2]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[3]  Chih-Jen Lin,et al.  On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization , 2007, IEEE Transactions on Neural Networks.

[4]  Jérôme Idier,et al.  Algorithms for Nonnegative Matrix Factorization with the β-Divergence , 2010, Neural Computation.

[5]  Inderjit S. Dhillon,et al.  Fast Newton-type Methods for the Least Squares Nonnegative Matrix Approximation Problem , 2007, SDM.

[6]  Andrzej Cichocki,et al.  Non-negative Matrix Factorization with Quasi-Newton Optimization , 2006, ICAISC.

[7]  Michael W. Berry,et al.  Algorithms and applications for approximate nonnegative matrix factorization , 2007, Comput. Stat. Data Anal..

[8]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[9]  Andrzej Cichocki,et al.  Nonnegative matrix factorization with quadratic programming , 2008, Neurocomputing.

[10]  Feng Qianjin,et al.  Projected gradient methods for Non-negative Matrix Factorization based relevance feedback algorithm in medical image retrieval , 2011 .

[11]  Andrzej Cichocki,et al.  Fast Nonnegative Matrix Factorization Algorithms Using Projected Gradient Approaches for Large-Scale Problems , 2008, Comput. Intell. Neurosci..

[12]  S. Sra.  Nonnegative Matrix Approximation: Algorithms and Applications , 2006 .