Rank-1 Tensor Approximation Methods and Application to Deflation

Because of the attractiveness of the canonical polyadic (CP) tensor decomposition in various applications, several algorithms have been designed to compute it, but efficient ones are still lacking. Iterative deflation algorithms based on successive rank-1 approximations can be used to perform this task, since the latter are rather easy to compute. We first present an algebraic rank-1 approximation method that performs better than the standard higher-order singular value decomposition (HOSVD) for three-way tensors. Second, we propose a new iterative rank-1 approximation algorithm that improves any other rank-1 approximation method. Third, we describe a probabilistic framework allowing to study the convergence of deflation CP decomposition (DCPD) algorithms based on successive rank-1 approximations. A set of computer experiments then validates theoretical results and demonstrates the efficiency of DCPD algorithms compared to other ones.

[1]  Pierre Comon,et al.  An iterative deflation algorithm for exact CP tensor decomposition , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[2]  Rasmus Bro,et al.  A comparison of algorithms for fitting the PARAFAC model , 2006, Comput. Stat. Data Anal..

[3]  Berkant Savas,et al.  Algorithms in data mining using matrix and tensor methods , 2008 .

[4]  Elijah Polak,et al.  Optimization: Algorithms and Consistent Approximations , 1997 .

[5]  André Lima Férrer de Almeida,et al.  PARAFAC-based unified tensor modeling for wireless communication systems with application to blind multiuser equalization , 2007, Signal Process..

[6]  Pierre Comon,et al.  Blind Separation of Instantaneous Mixtures of Dependent Sources , 2007, ICA.

[7]  Li Wang,et al.  Semidefinite Relaxations for Best Rank-1 Tensor Approximations , 2013, SIAM J. Matrix Anal. Appl..

[8]  Andrzej Cichocki,et al.  Tensor Deflation for CANDECOMP/PARAFAC— Part I: Alternating Subspace Update Algorithm , 2015, IEEE Transactions on Signal Processing.

[9]  David E. Booth,et al.  Multi-Way Analysis: Applications in the Chemical Sciences , 2005, Technometrics.

[10]  Pierre Comon,et al.  Handbook of Blind Source Separation: Independent Component Analysis and Applications , 2010 .

[11]  Pierre Comon,et al.  Subtracting a best rank-1 approximation may increase tensor rank , 2009, 2009 17th European Signal Processing Conference.

[12]  Berkant Savas,et al.  Quasi-Newton Methods on Grassmannians and Multilinear Approximations of Tensors , 2009, SIAM J. Sci. Comput..

[13]  Souleymen Sahnoun,et al.  Joint Source Estimation and Localization , 2015, IEEE Transactions on Signal Processing.

[14]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[15]  Jean B. Lasserre,et al.  Global Optimization with Polynomials and the Problem of Moments , 2000, SIAM J. Optim..

[16]  P. Comon,et al.  Symmetric tensor decomposition , 2009, 2009 17th European Signal Processing Conference.

[17]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[18]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .

[19]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[20]  J. Berge,et al.  Kruskal's polynomial for 2×2×2 arrays and a generalization to 2×n×n arrays , 1991 .

[21]  Bernard Mourrain,et al.  Border basis relaxation for polynomial optimization , 2016, J. Symb. Comput..

[22]  P. Paatero.  A weighted non-negative least squares algorithm for three-way ‘PARAFAC’ factor analysis , 1997 .

[23]  Gene H. Golub,et al.  Matrix computations , 1983 .

[24]  Luke Oeding,et al.  Eigenvectors of tensors and algorithms for Waring decomposition , 2011, J. Symb. Comput..

[25]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[26]  H. Kiers.  Towards a standardized notation and terminology in multiway analysis , 2000 .

[27]  Lieven De Lathauwer,et al.  Optimization-Based Algorithms for Tensor Decompositions: Canonical Polyadic Decomposition, Decomposition in Rank-(Lr, Lr, 1) Terms, and a New Generalization , 2013, SIAM J. Optim..

[28]  Liqi Wang,et al.  On the Global Convergence of the Alternating Least Squares Method for Rank-One Approximation to Generic Tensors , 2014, SIAM J. Matrix Anal. Appl..

[29]  F Wendling,et al.  EEG extended source localization: Tensor-based vs. conventional methods , 2014, NeuroImage.

[30]  G. Golub,et al.  Tracking a few extreme singular values and vectors in signal processing , 1990, Proc. IEEE.

[31]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[32]  Andrzej Cichocki,et al.  Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations , 2009, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[33]  P. Comon,et al.  Subtracting a best rank-1 approximation does not necessarily decrease tensor rank , 2015 .

[34]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[35]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[36]  C. Loan,et al.  Approximation with Kronecker Products , 1992 .