Approximate Matrix and Tensor Diagonalization by Unitary Transformations: Convergence of Jacobi-Type Algorithms

We propose a gradient-based Jacobi algorithm for a class of maximization problems on the unitary group, with a focus on approximate diagonalization of complex matrices and tensors by unitary transformations. We provide weak convergence results, and prove local linear convergence of this algorithm.The convergence results also apply to the case of real-valued tensors.

[1]  T. Rapcsák.  Geodesic convexity in nonlinear optimization , 1991 .

[2]  J. Mawhin,et al.  Nondegenerate Critical Manifolds , 1989 .

[3]  Antoine Souloumiac,et al.  Jacobi Angles for Simultaneous Diagonalization , 1996, SIAM J. Matrix Anal. Appl..

[4]  V. Hari,et al.  Convergence of the Cyclic and Quasi-cyclic Block Jacobi Methods , 2014, 1604.05825.

[5]  A. Bunse-Gerstner,et al.  Numerical Methods for Simultaneous Diagonalization , 1993, SIAM J. Matrix Anal. Appl..

[6]  Pierre Comon,et al.  From source separation to blind equalization, contrast-based approaches , 2001 .

[7]  Guoyin Li,et al.  Convergence rate analysis for the higher order power method in best rank one approximations of tensors , 2018, Numerische Mathematik.

[8]  Pierre-Antoine Absil,et al.  Quotient Geometry with Simple Geodesics for the Manifold of Fixed-Rank Positive-Semidefinite Matrices , 2020, SIAM J. Matrix Anal. Appl..

[9]  Susan A. Murphy,et al.  Monographs on statistics and applied probability , 1990 .

[10]  Bamdev Mishra,et al.  Manopt, a matlab toolbox for optimization on manifolds , 2013, J. Mach. Learn. Res..

[11]  Walter F. Mascarenhas,et al.  On the Convergence of the Jacobi Method for Arbitrary Orderings , 1995, SIAM J. Matrix Anal. Appl..

[12]  B. A. D. H. Brandwood.  A complex gradient operator and its applica-tion in adaptive array theory , 1983 .

[13]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[14]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[15]  P. McCullagh.  Tensor Methods in Statistics , 1987 .

[16]  Robert E. Mahony,et al.  An Extrinsic Look at the Riemannian Hessian , 2013, GSI.

[17]  Reinhold Schneider,et al.  Convergence Results for Projected Line-Search Methods on Varieties of Low-Rank Matrices Via Łojasiewicz Inequality , 2014, SIAM J. Optim..

[18]  Augustin Banyaga,et al.  A proof of the Morse-Bott Lemma , 2004 .

[19]  Pierre Comon,et al.  Globally convergent Jacobi-type algorithms for simultaneous orthogonal symmetric tensor diagonalization , 2017, SIAM J. Matrix Anal. Appl..

[20]  Paul Van Dooren,et al.  Jacobi Algorithm for the Best Low Multilinear Rank Approximation of Symmetric Tensors , 2013, SIAM J. Matrix Anal. Appl..

[21]  Are Hjørungnes,et al.  Complex-Valued Matrix Differentiation: Techniques and Key Results , 2007, IEEE Transactions on Signal Processing.

[22]  Levent Tunçel,et al.  Optimization algorithms on matrix manifolds , 2009, Math. Comput..

[23]  Visa Koivunen,et al.  Steepest Descent Algorithms for Optimization Under Unitary Matrix Constraint , 2008, IEEE Transactions on Signal Processing.

[24]  S. Krantz.  Function theory of several complex variables , 1982 .

[25]  Uwe Helmke,et al.  Jacobi's Algorithm on Compact Lie Algebras , 2004, SIAM J. Matrix Anal. Appl..

[26]  P. Feehan.  Optimal Łojasiewicz–Simon inequalities and Morse–Bott Yang–Mills energy functions , 2017, 1706.09349.

[27]  Alan Edelman,et al.  The Geometry of Algorithms with Orthogonality Constraints , 1998, SIAM J. Matrix Anal. Appl..

[28]  S. Helgason.  Differential Geometry, Lie Groups, and Symmetric Spaces , 1978 .

[29]  P. Absil,et al.  Erratum to: ``Global rates of convergence for nonconvex optimization on manifolds'' , 2016, IMA Journal of Numerical Analysis.

[30]  B. Hall.  Lie Groups, Lie Algebras, and Representations: An Elementary Introduction , 2004 .

[31]  Anima Anandkumar,et al.  Tensor decompositions for learning latent variable models , 2012, J. Mach. Learn. Res..

[32]  Jiawang Nie,et al.  Hermitian Tensor Decompositions , 2020, SIAM J. Matrix Anal. Appl..

[33]  V. Hari,et al.  On the convergence of complex Jacobi methods , 2018, Linear and Multilinear Algebra.

[34]  A. Uschmajew,et al.  A new convergence proof for the higher-order power method and generalizations , 2014, 1407.4586.

[35]  P. Comon.  Contrasts, independent component analysis, and blind deconvolution , 2004 .

[36]  Bruno Emile,et al.  Estimation of time delays with fewer sensors than sources , 1998, IEEE Trans. Signal Process..

[37]  L. Lathauwer,et al.  Signal Processing based on Multilinear Algebra , 1997 .

[38]  Pierre Comon,et al.  Handbook of Blind Source Separation: Independent Component Analysis and Applications , 2010 .

[39]  LAPACK Working.  A global convergence proof of cyclic Jacobi methods with block rotations , 2007 .

[40]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[41]  Robert E. Mahony,et al.  Convergence of the Iterates of Descent Methods for Analytic Cost Functions , 2005, SIAM J. Optim..

[42]  Harold R. Parks,et al.  A Primer of Real Analytic Functions , 1992 .

[43]  Shuzhong Zhang,et al.  Characterizing Real-Valued Multivariate Complex Polynomials and Their Symmetric Tensor Representations , 2015, SIAM J. Matrix Anal. Appl..