A further improvement of a fast damped Gauss-Newton algorithm for candecomp-parafac tensor decomposition

In this paper, a novel implementation of the damped Gauss-Newton algorithm (also known as Levenberg-Marquart) for the CANDECOMP-PARAFAC (CP) tensor decomposition is proposed. The method is based on a fast inversion of the approximate Hessian for the problem. It is shown that the inversion can be computed on O(NR6) operations, where N and R is the tensor order and rank, respectively. It is less than in the best existing state-of-the art algorithm with O(N3R6) operations. The damped Gauss-Newton algorithm is suitable namely for difficult scenarios, where nearly-colinear factors appear in several modes simultaneously. Performance of the method is shown on decomposition of large tensors (100 × 100 × 100 and 100 × 100 × 100 × 100) of rank 5 to 90.

[1]  Andrzej Cichocki,et al.  On Fast Computation of Gradients for CANDECOMP/PARAFAC Algorithms , 2012, ArXiv.

[2]  Rasmus Bro,et al.  A comparison of algorithms for fitting the PARAFAC model , 2006, Comput. Stat. Data Anal..

[3]  Rasmus Bro,et al.  The N-way Toolbox for MATLAB , 2000 .

[4]  Lieven De Lathauwer,et al.  Tensor-based techniques for the blind separation of DS-CDMA signals , 2007, Signal Process..

[5]  R. Bro,et al.  Practical aspects of PARAFAC modeling of fluorescence excitation‐emission data , 2003 .

[6]  Nicholas D. Sidiropoulos,et al.  Parafac techniques for signal separation , 2000 .

[7]  Tamara G. Kolda,et al.  Temporal Link Prediction Using Matrix and Tensor Factorizations , 2010, TKDD.

[8]  Chikio Hayashi,et al.  A NEW ALGORITHM TO SOLVE PARAFAC-MODEL , 1982 .

[9]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[10]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[11]  Laurent Albera,et al.  Multi-way space-time-wave-vector analysis for EEG source separation , 2012, Signal Process..

[12]  F. Chinesta,et al.  Recent advances on the use of separated representations , 2009 .

[13]  Andrzej Cichocki,et al.  Low Complexity Damped Gauss-Newton Algorithms for CANDECOMP/PARAFAC , 2012, SIAM J. Matrix Anal. Appl..

[14]  Lars Kai Hansen,et al.  Parallel Factor Analysis as an exploratory tool for wavelet transformed event-related EEG , 2006, NeuroImage.

[15]  D. D. Morrison Methods for nonlinear least squares problems and convergence proofs , 1960 .

[16]  Michael W. Berry,et al.  Discussion Tracking in Enron Email using PARAFAC. , 2008 .

[17]  Rasmus Bro,et al.  MULTI-WAY ANALYSIS IN THE FOOD INDUSTRY Models, Algorithms & Applications , 1998 .

[18]  Ben C. Mitchell,et al.  Slowly converging parafac sequences: Swamps and two‐factor degeneracies , 1994 .

[19]  Ilghiz Ibraghimov,et al.  Application of the three‐way decomposition for matrix compression , 2002, Numer. Linear Algebra Appl..

[20]  P. Paatero Least squares formulation of robust non-negative factor analysis , 1997 .

[21]  Tamir Hazan,et al.  Non-negative tensor factorization with applications to statistics and computer vision , 2005, ICML.

[22]  Zbynek Koldovský,et al.  Simultaneous search for all modes in multilinear models , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[23]  P. Paatero A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[24]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..