Jacobi-type algorithm for low rank orthogonal approximation of symmetric tensors and its convergence analysis

In this paper, we propose a Jacobi-type algorithm to solve the low rank orthogonal approximation problem of symmetric tensors. This algorithm includes as a special case the well-known Jacobi CoM2 algorithm for the approximate orthogonal diagonalization problem of symmetric tensors. We first prove the weak convergence of this algorithm, \textit{i.e.} any accumulation point is a stationary point. Then we study the global convergence of this algorithm under a gradient based ordering for a special case: the best rank-2 orthogonal approximation of 3rd order symmetric tensors, and prove that an accumulation point is the unique limit point under some conditions. Numerical experiments are presented to show the efficiency of this algorithm.

[1]  Pierre Comon,et al.  Globally convergent Jacobi-type algorithms for simultaneous orthogonal symmetric tensor diagonalization , 2017, SIAM J. Matrix Anal. Appl..

[2]  Pierre Comon,et al.  Approximate Matrix and Tensor Diagonalization by Unitary Transformations: Convergence of Jacobi-Type Algorithms , 2019, SIAM J. Optim..

[3]  Tamara G. Kolda,et al.  Shifted Power Method for Computing Tensor Eigenpairs , 2010, SIAM J. Matrix Anal. Appl..

[4]  Gene H. Golub,et al.  Rank-One Approximation to High Order Tensors , 2001, SIAM J. Matrix Anal. Appl..

[5]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[6]  Gene H. Golub,et al.  Symmetric Tensors and Symmetric Tensor Rank , 2008, SIAM J. Matrix Anal. Appl..

[7]  J. Bell.  The Bernstein and Nikolsky inequalities for trigonometric polynomials , 2015 .

[8]  Pierre Comon,et al.  Tensors : A brief introduction , 2014, IEEE Signal Processing Magazine.

[9]  Bamdev Mishra,et al.  Manopt, a matlab toolbox for optimization on manifolds , 2013, J. Mach. Learn. Res..

[10]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[11]  Yousef Saad,et al.  On the Tensor SVD and the Optimal Low Rank Orthogonal Approximation of Tensors , 2008, SIAM J. Matrix Anal. Appl..

[12]  L. Qi,et al.  Tensor Analysis: Spectral Theory and Special Tensors , 2017 .

[13]  Haiping Lu,et al.  Multilinear Subspace Learning: Dimensionality Reduction of Multidimensional Data , 2013 .

[14]  Phillip A. Regalia,et al.  On the Best Rank-1 Approximation of Higher-Order Supersymmetric Tensors , 2001, SIAM J. Matrix Anal. Appl..

[15]  Tamara G. Kolda,et al.  Orthogonal Tensor Decompositions , 2000, SIAM J. Matrix Anal. Appl..

[16]  Michael K. Ng,et al.  Symmetric orthogonal approximation to symmetric tensors with applications to image reconstruction , 2018, Numer. Linear Algebra Appl..

[17]  L. Lathauwer,et al.  On the Best Rank-1 and Rank-( , 2004 .

[18]  L. Lathauwer,et al.  Signal Processing based on Multilinear Algebra , 1997 .

[19]  Robert E. Mahony,et al.  Optimization Algorithms on Matrix Manifolds , 2007 .

[20]  Reinhold Schneider,et al.  Convergence Results for Projected Line-Search Methods on Varieties of Low-Rank Matrices Via Łojasiewicz Inequality , 2014, SIAM J. Optim..

[21]  Chen Ling,et al.  The Best Rank-1 Approximation of a Symmetric Tensor and Related Spherical Optimization Problems , 2012, SIAM J. Matrix Anal. Appl..

[22]  P. Comon.  Tensor Diagonalization, A useful Tool in Signal Processing , 1994 .

[23]  Paul Van Dooren,et al.  Jacobi Algorithm for the Best Low Multilinear Rank Approximation of Symmetric Tensors , 2013, SIAM J. Matrix Anal. Appl..

[24]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[25]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[26]  Pierre Comon,et al.  A Finite Algorithm to Compute Rank-1 Tensor Approximations , 2016, IEEE Signal Processing Letters.

[27]  P. Comon.  Independent Component Analysis , 1992 .

[28]  Nikos D. Sidiropoulos,et al.  Tensor Decomposition for Signal Processing and Machine Learning , 2016, IEEE Transactions on Signal Processing.

[29]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[30]  Fei Wang,et al.  Z-eigenvalue methods for a global polynomial optimization problem , 2009, Math. Program..

[31]  Yinyu Ye,et al.  The cubic spherical optimization problems , 2012, Math. Comput..

[32]  P. Comon,et al.  On approximate diagonalization of third order symmetric tensors by orthogonal transformations , 2018, Linear Algebra and its Applications.