Computing the polyadic decomposition of nonnegative third order tensors

Computing the minimal polyadic decomposition (also often referred to as canonical decomposition or sometimes Parafac) amounts to finding the global minimum of a coercive polynomial in many variables. In the case of arrays with nonnegative entries, the low-rank approximation problem is well posed. In addition, due to the large dimension of the problem, the decomposition can be rather efficiently calculated with the help of preconditioned nonlinear conjugate gradient algorithms, as subsequently shown, if equipped with an algebraic calculation of the globally optimal stepsize in low dimension. Other algorithms are also studied (gradient and quasi-Newton approaches) for comparisons. Two versions of each algorithm are considered: the enhanced line search version (ELS), and the backtracking version alternating with ELS. Computer simulations are provided and demonstrate the good behavior of these algorithms dedicated to nonnegative arrays, compared to others put forward in the literature. Finally, applications in the context of data analysis illustrate various algorithms. The main advantage of the suggested approach is to explicitly take into account the nonnegative nature of the loading matrices in the problem parameterization, instead of enforcing positive entries by projection. According to the experiments we have run, such an approach also happens to be more robust with respect to possible modeling errors.

[1]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[2]  Yang Li,et al.  Alternating coupled vectors resolution (ACOVER) method for trilinear analysis of three‐way data , 1999 .

[3]  C. Garnier,et al.  Copper complexing properties of dissolved organic matter: PARAFAC treatment of fluorescence quenching , 2011 .

[4]  Bijan Afsari,et al.  Sensitivity Analysis for the Problem of Matrix Joint Diagonalization , 2008, SIAM J. Matrix Anal. Appl..

[5]  Pierre Comon,et al.  Tensors versus matrices usefulness and unexpected properties , 2009, 2009 IEEE/SP 15th Workshop on Statistical Signal Processing.

[6]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics , 2019, Wiley Series in Probability and Statistics.

[7]  Elijah Polak,et al.  Optimization: Algorithms and Consistent Approximations , 1997 .

[8]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[9]  R. Bro,et al.  PARAFAC and missing values , 2005 .

[10]  P. Paatero A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[11]  Rasmus Bro,et al.  MULTI-WAY ANALYSIS IN THE FOOD INDUSTRY Models, Algorithms & Applications , 1998 .

[12]  R. Bro PARAFAC. Tutorial and applications , 1997 .

[13]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[14]  P. Paatero The Multilinear Engine—A Table-Driven, Least Squares Program for Solving Multilinear Problems, Including the n-Way Parallel Factor Analysis Model , 1999 .

[15]  A. Franc Etude algébrique des multitableaux : apports de l'algèbre tensorielle , 1992 .

[16]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[17]  Qiang Zhang,et al.  Tensor methods for hyperspectral data analysis: a space object material identification study. , 2008, Journal of the Optical Society of America. A, Optics, image science, and vision.

[18]  Pierre Comon,et al.  Nonnegative approximations of nonnegative tensors , 2009, ArXiv.

[19]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[20]  J. Shewchuk An Introduction to the Conjugate Gradient Method Without the Agonizing Pain , 1994 .

[21]  P. Paatero Construction and analysis of degenerate PARAFAC models , 2000 .

[22]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[23]  Rasmus Bro,et al.  A comparison of algorithms for fitting the PARAFAC model , 2006, Comput. Stat. Data Anal..

[24]  Driss Aboutajdine,et al.  Gradient-based joint block diagonalization algorithms: Application to blind separation of FIR convolutive mixtures , 2010, Signal Process..

[25]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .

[26]  Michael Clausen,et al.  Typical Tensorial Rank , 1997 .

[27]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[28]  F. L. Hitchcock The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[29]  C. G. Bollini,et al.  On the Reduction Formula of Feinberg and Pais , 1965 .

[30]  R. Redon,et al.  A simple correction method of inner filter effects affecting FEEM and its application to the PARAFAC decomposition , 2009 .

[31]  David E. Booth,et al.  Multi-Way Analysis: Applications in the Chemical Sciences , 2005, Technometrics.

[32]  D. Luenberger Optimization by Vector Space Methods , 1968 .

[33]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics (Revised Edition) , 1999 .