Non-orthogonal tensor diagonalization

Tensor diagonalization means transforming a given tensor to an exactly or nearly diagonal form through multiplying the tensor by non-orthogonal invertible matrices along selected dimensions of the tensor. It is generalization of approximate joint diagonalization (AJD) of a set of matrices. In particular, we derive (1) a new algorithm for symmetric AJD, which is called two-sided symmetric diagonalization of order-three tensor, (2) a similar algorithm for non-symmetric AJD, also called general two-sided diagonalization of an order-3 tensor, and (3) an algorithm for three-sided diagonalization of order-3 or order-4 tensors. The latter two algorithms may serve for canonical polyadic (CP) tensor decomposition, and they can outperform other CP tensor decomposition methods in terms of computational speed under the restriction that the tensor rank does not exceed the tensor multilinear rank. Finally, we propose (4) similar algorithms for tensor block diagonalization, which is related to the tensor block-term decomposition.

[1]  Alwin Stegeman,et al.  Candecomp/Parafac: From Diverging Components to a Decomposition in Block Terms , 2012, SIAM J. Matrix Anal. Appl..

[2]  A. Stegeman,et al.  On the Non-Existence of Optimal Solutions and the Occurrence of “Degeneracy” in the CANDECOMP/PARAFAC Model , 2008, Psychometrika.

[3]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[4]  Andrzej Cichocki,et al.  Partitioned Alternating Least Squares Technique for Canonical Polyadic Tensor Decomposition , 2016, IEEE Signal Processing Letters.

[5]  Andrzej Cichocki,et al.  Low rank tensor deconvolution , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  P. Comon,et al.  TENSOR DIAGONALIZATION BY ORTHOGONAL TRANSFORMS , 2007 .

[7]  Eric Moreau,et al.  Fast Jacobi algorithm for non-orthogonal joint diagonalization of non-symmetric third-order tensors , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[8]  Florian Roemer,et al.  A semi-algebraic framework for approximate CP decompositions via simultaneous matrix diagonalizations (SECSI) , 2013, Signal Process..

[9]  E. Cuthill,et al.  Reducing the bandwidth of sparse symmetric matrices , 1969, ACM '69.

[10]  Zbynek Koldovský,et al.  Algorithms for nonorthogonal approximate joint block-diagonalization , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[11]  Laurent Sorber,et al.  Data Fusion: Tensor Factorizations by Complex Optimization (Data fusie: tensor factorisaties d.m.v. complexe optimalisatie) , 2014 .

[12]  Antoine Souloumiac,et al.  Nonorthogonal Joint Diagonalization by Combining Givens and Hyperbolic Rotations , 2009, IEEE Transactions on Signal Processing.

[13]  XIJING GUO,et al.  Uni-mode and Partial Uniqueness Conditions for CANDECOMP/PARAFAC of Three-Way Arrays with Linearly Dependent Loadings , 2012, SIAM J. Matrix Anal. Appl..

[14]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[15]  Andrzej Cichocki,et al.  Tensor diagonalization - a new tool for PARAFAC and block-term decomposition , 2014, ArXiv.

[16]  Martin Haardt,et al.  Extension of the semi-algebraic framework for approximate CP decompositions via non-symmetric simultaneous matrix diagonalization , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[17]  Arie Yeredor,et al.  On Computation of Approximate Joint Block-Diagonalization Using Ordinary AJD , 2012, LVA/ICA.

[18]  P. Kroonenberg Applied Multiway Data Analysis , 2008 .

[19]  J. Kruskal Rank, decomposition, and uniqueness for 3-way and n -way arrays , 1989 .

[20]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[21]  P. Comon Tensor Diagonalization, A useful Tool in Signal Processing , 1994 .

[22]  Bijan Afsari,et al.  Sensitivity Analysis for the Problem of Matrix Joint Diagonalization , 2008, SIAM J. Matrix Anal. Appl..

[23]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part III: Alternating Least Squares Algorithms , 2008, SIAM J. Matrix Anal. Appl..

[24]  Andrzej Cichocki,et al.  Fast Alternating LS Algorithms for High Order CANDECOMP/PARAFAC Tensor Factorizations , 2013, IEEE Transactions on Signal Processing.

[25]  Lieven De Lathauwer,et al.  A link between the decomposition of a third-order tensor in rank-(L,L,1) terms and Joint Block Diagonalization , 2009, 2009 3rd IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).

[26]  Shufang Xu,et al.  A Matrix Polynomial Spectral Approach for General Joint Block Diagonalization , 2015, SIAM J. Matrix Anal. Appl..

[27]  Rasmus Bro,et al.  MULTI-WAY ANALYSIS IN THE FOOD INDUSTRY Models, Algorithms & Applications , 1998 .

[28]  Eric Moreau,et al.  Joint Matrices Decompositions and Blind Source Separation , 2014 .

[29]  J. Kruskal Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[30]  Fabian J. Theis,et al.  Towards a general independent subspace analysis , 2006, NIPS.

[31]  Zbynek Koldovský,et al.  Optimal pairing of signal components separated by blind techniques , 2004, IEEE Signal Processing Letters.

[32]  Zbynek Koldovský,et al.  Weight Adjusted Tensor Method for Blind Separation of Underdetermined Mixtures of Nonstationary Sources , 2011, IEEE Transactions on Signal Processing.

[33]  Joos Vandewalle,et al.  Computation of the Canonical Decomposition by Means of a Simultaneous Generalized Schur Decomposition , 2005, SIAM J. Matrix Anal. Appl..

[34]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .

[35]  Andrzej Cichocki,et al.  CANDECOMP/PARAFAC Decomposition of High-Order Tensors Through Tensor Reshaping , 2012, IEEE Transactions on Signal Processing.

[36]  Andrzej Cichocki,et al.  On Fast algorithms for orthogonal Tucker decomposition , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[37]  Arie Yeredor,et al.  Joint Matrices Decompositions and Blind Source Separation: A survey of methods, identification, and applications , 2014, IEEE Signal Processing Magazine.

[38]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[39]  De LathauwerLieven Decompositions of a Higher-Order Tensor in Block TermsPart II: Definitions and Uniqueness , 2008 .

[40]  Hagit Messer,et al.  Second-Order Multidimensional ICA: Performance Analysis , 2012, IEEE Transactions on Signal Processing.

[41]  Jianhong Wu,et al.  Data clustering - theory, algorithms, and applications , 2007 .

[42]  Pierre Comon,et al.  Approximate tensor diagonalization by invertible transforms , 2009, 2009 17th European Signal Processing Conference.

[43]  Nikos D. Sidiropoulos,et al.  Kruskal's permutation lemma and the identification of CANDECOMP/PARAFAC and bilinear models with constant modulus constraints , 2004, IEEE Transactions on Signal Processing.

[44]  Daniel Kressner,et al.  A literature survey of low‐rank tensor approximation techniques , 2013, 1302.7121.

[45]  Dimitri Nion,et al.  A Tensor Framework for Nonunitary Joint Block Diagonalization , 2011, IEEE Transactions on Signal Processing.

[46]  P. Tichavsky,et al.  Fast Approximate Joint Diagonalization Incorporating Weight Matrices , 2009, IEEE Transactions on Signal Processing.

[47]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part II: Definitions and Uniqueness , 2008, SIAM J. Matrix Anal. Appl..

[48]  David E. Booth,et al.  Multi-Way Analysis: Applications in the Chemical Sciences , 2005, Technometrics.

[49]  Hicham Ghennioui,et al.  A Nonunitary Joint Block Diagonalization Algorithm for Blind Separation of Convolutive Mixtures of Sources , 2007, IEEE Signal Processing Letters.

[50]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[51]  A. Stegeman,et al.  On Kruskal's uniqueness condition for the Candecomp/Parafac decomposition , 2007 .

[52]  Yousef Saad,et al.  On the Tensor SVD and the Optimal Low Rank Orthogonal Approximation of Tensors , 2008, SIAM J. Matrix Anal. Appl..

[53]  Lieven De Lathauwer,et al.  A Block Component Model-Based Blind DS-CDMA Receiver , 2008, IEEE Transactions on Signal Processing.

[54]  Andrzej Cichocki,et al.  Two-sided diagonalization of order-three tensors , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[55]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[56]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[57]  R. Harshman,et al.  Modeling multi‐way data with linearly dependent loadings , 2009 .

[58]  W. Marsden I and J , 2012 .

[59]  Zbynek Koldovský,et al.  Cramér-Rao-Induced Bounds for CANDECOMP/PARAFAC Tensor Decomposition , 2012, IEEE Transactions on Signal Processing.

[60]  B. Kowalski,et al.  Tensorial resolution: A direct trilinear decomposition , 1990 .

[61]  De LathauwerLieven,et al.  Decompositions of a Higher-Order Tensor in Block TermsPart III: Alternating Least Squares Algorithms , 2008 .

[62]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[63]  Hao Shen,et al.  A Block-Jacobi Algorithm for Non-Symmetric Joint Diagonalization of Matrices , 2015, LVA/ICA.