Error Preserving Correction for CPD and Bounded-Norm CPD

In CANDECOMP/PARAFAC tensor decomposition, degeneracy often occurs in some difficult scenarios, e.g., when the rank exceeds the tensor dimension, or when the loading components are highly collinear in several or all modes, or when CPD does not have an optimal solution. In such the cases, norms of some rank-1 terms become significantly large and cancel each other. This makes algorithms getting stuck in local minima while running a huge number of iterations does not improve the decomposition. In this paper, we propose an error preservation correction method to deal with such problem. Our aim is to seek a new tensor whose norms of rank-1 tensor components are minimised in an optimization problem, while it preserves the approximation error. An alternating correction algorithm and an all-atone algorithm have been developed for the problem. In addition, we propose a novel CPD with a bound constraint on a norm of the rank-one tensors. The method can be useful for decomposing tensors that cannot be analyzed by traditional algorithms, such as tensors corresponding to the matrix multiplication.

[1]  Andrzej Cichocki,et al.  Tensor Networks for Dimensionality Reduction and Large-scale Optimization: Part 1 Low-Rank Tensor Decompositions , 2016, Found. Trends Mach. Learn..

[2]  Paul T. Boggs,et al.  Sequential Quadratic Programming , 1995, Acta Numerica.

[3]  Alwin Stegeman,et al.  Candecomp/Parafac: From Diverging Components to a Decomposition in Block Terms , 2012, SIAM J. Matrix Anal. Appl..

[4]  W. Rayens,et al.  Two-factor degeneracies and a stabilization of PARAFAC , 1997 .

[5]  Andrzej Cichocki,et al.  Partitioned Hierarchical alternating least squares algorithm for CP tensor decomposition , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Andrzej Cichocki,et al.  A further improvement of a fast damped Gauss-Newton algorithm for candecomp-parafac tensor decomposition , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  A. Stegeman,et al.  On the Non-Existence of Optimal Solutions and the Occurrence of “Degeneracy” in the CANDECOMP/PARAFAC Model , 2008, Psychometrika.

[8]  P. Paatero.  Construction and analysis of degenerate PARAFAC models , 2000 .

[9]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[10]  Andrzej Cichocki,et al.  Fast Alternating LS Algorithms for High Order CANDECOMP/PARAFAC Tensor Factorizations , 2013, IEEE Transactions on Signal Processing.

[11]  Ben C. Mitchell,et al.  Slowly converging parafac sequences: Swamps and two‐factor degeneracies , 1994 .

[12]  Pierre Comon,et al.  Nonnegative approximations of nonnegative tensors , 2009, ArXiv.

[13]  Andrzej Cichocki,et al.  Low Complexity Damped Gauss-Newton Algorithms for CANDECOMP/PARAFAC , 2012, SIAM J. Matrix Anal. Appl..

[14]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[15]  R. A. Harshman,et al.  Data preprocessing and the extended PARAFAC model , 1984 .

[16]  Andrzej Cichocki,et al.  Numerical CP decomposition of some difficult tensors , 2016, J. Comput. Appl. Math..

[17]  J. Kruskal,et al.  A two-stage procedure incorporating good features of both trilinear and quadrilinear models , 1989 .

[18]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .

[19]  W. Gander,et al.  A constrained eigenvalue problem , 1988 .

[20]  Andrzej Cichocki,et al.  Partitioned Alternating Least Squares Technique for Canonical Polyadic Tensor Decomposition , 2016, IEEE Signal Processing Letters.

[21]  P. Paatero.  A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[22]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.