On the convergence of Jacobi-type algorithms for Independent Component Analysis

Jacobi-type algorithms for simultaneous approximate diagonalization of real (or complex) symmetric tensors have been widely used in independent component analysis (ICA) because of their good performance. One natural way of choosing the index pairs in Jacobitype algorithms is the classical cyclic ordering, while the other way is based on the Riemannian gradient in each iteration. In this paper, we mainly review in an accessible manner our recent results in a series of papers about weak and global convergence of these Jacobitype algorithms. These results are mainly based on the Lojasiewicz gradient inequality.

[1]  S. Łojasiewicz Ensembles semi-analytiques , 1965 .

[2]  Harold R. Parks,et al.  A Primer of Real Analytic Functions , 1992 .

[3]  S. Łojasiewicz Sur la géométrie semi- et sous- analytique , 1993 .

[4]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[5]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[6]  P. Comon Tensor Diagonalization, A useful Tool in Signal Processing , 1994 .

[7]  Antoine Souloumiac,et al.  Jacobi Angles for Simultaneous Diagonalization , 1996, SIAM J. Matrix Anal. Appl..

[8]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[9]  Joos Vandewalle,et al.  Blind source separation by simultaneous third-order tensor diagonalization , 1996, 1996 8th European Signal Processing Conference (EUSIPCO 1996).

[10]  L. Lathauwer,et al.  Signal Processing based on Multilinear Algebra , 1997 .

[11]  Bruno Emile,et al.  Estimation of time delays with fewer sensors than sources , 1998, IEEE Trans. Signal Process..

[12]  Joos Vandewalle,et al.  Independent component analysis and (simultaneous) third-order tensor diagonalization , 2001, IEEE Trans. Signal Process..

[13]  Pierre Comon,et al.  From source separation to blind equalization, contrast-based approaches , 2001 .

[14]  P. Comon Contrasts, independent component analysis, and blind deconvolution , 2004 .

[15]  Robert E. Mahony,et al.  Convergence of the Iterates of Descent Methods for Analytic Cost Functions , 2005, SIAM J. Optim..

[16]  Gene H. Golub,et al.  Symmetric Tensors and Symmetric Tensor Rank , 2008, SIAM J. Matrix Anal. Appl..

[17]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[18]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[19]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[20]  Paul Van Dooren,et al.  Jacobi Algorithm for the Best Low Multilinear Rank Approximation of Symmetric Tensors , 2013, SIAM J. Matrix Anal. Appl..

[21]  Pierre Comon,et al.  Tensors : A brief introduction , 2014, IEEE Signal Processing Magazine.

[22]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2014, Math. Program..

[23]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[24]  Reinhold Schneider,et al.  Convergence Results for Projected Line-Search Methods on Varieties of Low-Rank Matrices Via Łojasiewicz Inequality , 2014, SIAM J. Optim..

[25]  Shuzhong Zhang,et al.  Characterizing Real-Valued Multivariate Complex Polynomials and Their Symmetric Tensor Representations , 2015, SIAM J. Matrix Anal. Appl..

[26]  L. Qi,et al.  Tensor Analysis: Spectral Theory and Special Tensors , 2017 .

[27]  Nikos D. Sidiropoulos,et al.  Tensor Decomposition for Signal Processing and Machine Learning , 2016, IEEE Transactions on Signal Processing.

[28]  Pierre Comon,et al.  Globally convergent Jacobi-type algorithms for simultaneous orthogonal symmetric tensor diagonalization , 2017, SIAM J. Matrix Anal. Appl..

[29]  P. Comon,et al.  Jacobi-type algorithm for low rank orthogonal approximation of symmetric tensors and its convergence analysis , 2019, ArXiv.

[30]  P. Comon,et al.  On approximate diagonalization of third order symmetric tensors by orthogonal transformations , 2018, Linear Algebra and its Applications.

[31]  P. Absil,et al.  Erratum to: ``Global rates of convergence for nonconvex optimization on manifolds'' , 2016, IMA Journal of Numerical Analysis.

[32]  Pierre Comon,et al.  Approximate Matrix and Tensor Diagonalization by Unitary Transformations: Convergence of Jacobi-Type Algorithms , 2019, SIAM J. Optim..

[33]  Jiawang Nie,et al.  Hermitian Tensor Decompositions , 2019, SIAM J. Matrix Anal. Appl..