Best Rank-One Tensor Approximation and Parallel Update Algorithm for CPD

A novel algorithm is proposed for CANDECOMP/PARAFAC tensor decomposition to exploit best rank-1 tensor approximation. Different from the existing algorithms, our algorithm updates rank-1 tensors simultaneously in parallel. In order to achieve this, we develop new all-at-once algorithms for best rank-1 tensor approximation based on the Levenberg-Marquardt method and the rotational update. We show that the LM algorithm has the same complexity of first-order optimisation algorithms, while the rotational method leads to solving the best rank-1 approximation of tensors of size $2 \times 2 \times \cdots \times 2$. We derive a closed-form expression of the best rank-1 tensor of $2\times 2 \times 2$ tensors and present an ALS algorithm which updates 3 component at a time for higher order tensors. The proposed algorithm is illustrated in decomposition of difficult tensors which are associated with multiplication of two matrices.

[1]  Pierre Comon,et al.  A Finite Algorithm to Compute Rank-1 Tensor Approximations , 2016, IEEE Signal Processing Letters.

[2]  Andrzej Cichocki,et al.  Tensor Deflation for CANDECOMP/PARAFAC. Part 3: Rank Splitting , 2015, ArXiv.

[3]  V. Strassen Gaussian elimination is not optimal , 1969 .

[4]  P. Paatero A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[5]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[6]  Ivan Oseledets,et al.  Tensor-Train Decomposition , 2011, SIAM J. Sci. Comput..

[7]  Andrzej Cichocki,et al.  Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations , 2009, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[8]  Eugene E. Tyrtyshnikov,et al.  Breaking the Curse of Dimensionality, Or How to Use SVD in Many Dimensions , 2009, SIAM J. Sci. Comput..

[9]  Andrzej Cichocki,et al.  Error Preserving Correction for CPD and Bounded-Norm CPD , 2017, ArXiv.

[10]  Emmanuel J. Candès,et al.  Templates for convex cone problems with applications to sparse signal recovery , 2010, Math. Program. Comput..

[11]  Andrzej Cichocki,et al.  Low Complexity Damped Gauss-Newton Algorithms for CANDECOMP/PARAFAC , 2012, SIAM J. Matrix Anal. Appl..

[12]  Andrzej Cichocki,et al.  Numerical CP decomposition of some difficult tensors , 2016, J. Comput. Appl. Math..

[13]  P. Paatero,et al.  Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values† , 1994 .

[14]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[15]  Stephen P. Boyd,et al.  Proximal Algorithms , 2013, Found. Trends Optim..

[16]  H. B. Nielsen DAMPING PARAMETER IN MARQUARDT ’ S METHOD , 1999 .

[17]  G. Vidal Efficient classical simulation of slightly entangled quantum computations. , 2003, Physical review letters.

[18]  Andrzej Cichocki,et al.  Tensor Deflation for CANDECOMP/PARAFAC— Part I: Alternating Subspace Update Algorithm , 2015, IEEE Transactions on Signal Processing.

[19]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[20]  Lieven De Lathauwer,et al.  Structured Data Fusion , 2015, IEEE Journal of Selected Topics in Signal Processing.

[21]  K. Meerbergen,et al.  On the truncated multilinear singular value decomposition , 2011 .

[22]  Andrzej Cichocki,et al.  Fast Alternating LS Algorithms for High Order CANDECOMP/PARAFAC Tensor Factorizations , 2013, IEEE Transactions on Signal Processing.

[23]  Daniel M. Dunlavy,et al.  A scalable optimization approach for fitting canonical tensor decompositions , 2011 .

[24]  Andrzej Cichocki,et al.  Tensor Networks for Latent Variable Analysis. Part I: Algorithms for Tensor Train Decomposition , 2016, ArXiv.

[25]  Andrzej Cichocki,et al.  Partitioned Hierarchical alternating least squares algorithm for CP tensor decomposition , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[26]  Andrzej Cichocki,et al.  Partitioned Alternating Least Squares Technique for Canonical Polyadic Tensor Decomposition , 2016, IEEE Signal Processing Letters.