Tensors : A brief introduction

Tensor decompositions are at the core of many blind source separation (BSS) algorithms, either explicitly or implicitly. In particular, the canonical polyadic (CP) tensor decomposition plays a central role in the identification of underdetermined mixtures. Despite some similarities, CP and singular value decomposition (SVD) are quite different. More generally, tensors and matrices enjoy different properties, as pointed out in this brief introduction.

[1]  J. Berge,et al.  Kruskal's polynomial for 2×2×2 arrays and a generalization to 2×n×n arrays , 1991 .

[2]  A. Stegeman,et al.  On Kruskal's uniqueness condition for the Candecomp/Parafac decomposition , 2007 .

[3]  Gian-Carlo Rota,et al.  Apolarity and Canonical Forms for Homogeneous Polynomials , 1993, Eur. J. Comb..

[4]  Andreas Ziehe,et al.  A Fast Algorithm for Joint Diagonalization with Non-orthogonal Transformations and its Application to Blind Source Separation , 2004, J. Mach. Learn. Res..

[5]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[6]  Rasmus Bro,et al.  A comparison of algorithms for fitting the PARAFAC model , 2006, Comput. Stat. Data Anal..

[7]  Susan A. Murphy,et al.  Monographs on statistics and applied probability , 1990 .

[8]  J. Landsberg.  Tensors: Geometry and Applications , 2011 .

[9]  Lieven De Lathauwer,et al.  Optimization-Based Algorithms for Tensor Decompositions: Canonical Polyadic Decomposition, Decomposition in Rank-(Lr, Lr, 1) Terms, and a New Generalization , 2013, SIAM J. Optim..

[10]  Luke Oeding,et al.  Eigenvectors of tensors and algorithms for Waring decomposition , 2011, J. Symb. Comput..

[11]  E. C. Ron,et al.  From Vectors to Tensors , 2005 .

[12]  Elina Robeva.  Orthogonal Tensor Decomposition , 2014 .

[13]  Pierre Comon,et al.  Handbook of Blind Source Separation: Independent Component Analysis and Applications , 2010 .

[14]  Lek-Heng Lim,et al.  Singular values and eigenvalues of tensors: a variational approach , 2005, 1st IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, 2005..

[15]  Yiju Wang,et al.  On the best rank-1 approximation to higher-order symmetric tensors , 2007, Math. Comput. Model..

[16]  Anthony Iarrobino,et al.  Inverse System of a Symbolic Power II. The Waring Problem for Forms , 1995 .

[17]  P. Comon,et al.  Generic and typical ranks of multi-way arrays , 2009 .

[18]  Xi-Lin Li,et al.  Nonorthogonal Joint Diagonalization Free of Degenerate Solution , 2007, IEEE Transactions on Signal Processing.

[19]  Lieven De Lathauwer,et al.  A Method to Avoid Diverging Components in the Candecomp/Parafac Model for Generic I˟J˟2 Arrays , 2008, SIAM J. Matrix Anal. Appl..

[20]  Pierre Comon,et al.  Subtracting a best rank-1 approximation may increase tensor rank , 2009, 2009 17th European Signal Processing Conference.

[21]  Lieven De Lathauwer,et al.  On the Uniqueness of the Canonical Polyadic Decomposition of Third-Order Tensors - Part II: Uniqueness of the Overall Decomposition , 2013, SIAM J. Matrix Anal. Appl..

[22]  Nikos D. Sidiropoulos,et al.  Kruskal's permutation lemma and the identification of CANDECOMP/PARAFAC and bilinear models with constant modulus constraints , 2004, IEEE Transactions on Signal Processing.

[23]  P. Comon,et al.  ICAR: a tool for blind source separation using fourth-order statistics only , 2005, IEEE Transactions on Signal Processing.

[24]  P. Regalia,et al.  Tensor Approximation and Signal Processing Applications , 2005 .

[25]  Gene H. Golub,et al.  Symmetric Tensors and Symmetric Tensor Rank , 2008, SIAM J. Matrix Anal. Appl..

[26]  P. Paatero.  Construction and analysis of degenerate PARAFAC models , 2000 .

[27]  P. Comon,et al.  Subtracting a best rank-1 approximation does not necessarily decrease tensor rank , 2015 .

[28]  Andrzej Cichocki,et al.  Tensor Decompositions for Signal Processing Applications: From two-way to multiway component analysis , 2014, IEEE Signal Processing Magazine.

[29]  Lieven De Lathauwer,et al.  Blind Separation of Exponential Polynomials and the Decomposition of a Tensor in Rank-(Lr, Lr, 1) Terms , 2011, SIAM J. Matrix Anal. Appl..

[30]  P. McCullagh.  Tensor Methods in Statistics , 1987 .

[31]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[32]  Westone,et al.  Home Page , 2004, 2022 2nd International Conference on Intelligent Cybernetics Technology & Applications (ICICyTA).

[33]  J. Berge.  Simplicity and typical rank of three-way arrays, with applications to Tucker-3 analysis with simple cores , 2004 .

[34]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[35]  Grigoriy Blekherman,et al.  Typical Real Ranks of Binary Forms , 2012, Found. Comput. Math..

[36]  P. Comon,et al.  On the typical rank of real binary forms , 2009, 0909.4865.

[37]  Joos Vandewalle,et al.  Independent component analysis and (simultaneous) third-order tensor diagonalization , 2001, IEEE Trans. Signal Process..

[38]  S. Friedland.  Best rank one approximation of real symmetric tensors can be chosen symmetric , 2011, 1110.5689.

[39]  Arnold Schönhage,et al.  Partial and Total Matrix Multiplication , 1981, SIAM J. Comput..

[40]  Michael Clausen,et al.  Typical Tensorial Rank , 1997 .

[41]  Chen Ling,et al.  The Best Rank-1 Approximation of a Symmetric Tensor and Related Spherical Optimization Problems , 2012, SIAM J. Matrix Anal. Appl..

[42]  Giorgio Ottaviani,et al.  On Generic Identifiability of 3-Tensors of Small Rank , 2011, SIAM J. Matrix Anal. Appl..

[43]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[44]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[45]  Arie Yeredor,et al.  Joint Matrices Decompositions and Blind Source Separation: A survey of methods, identification, and applications , 2014, IEEE Signal Processing Magazine.

[46]  H. Kiers.  Towards a standardized notation and terminology in multiway analysis , 2000 .

[47]  Phillip A. Regalia,et al.  On the Best Rank-1 Approximation of Higher-Order Supersymmetric Tensors , 2001, SIAM J. Matrix Anal. Appl..

[48]  Tamara G. Kolda,et al.  Orthogonal Tensor Decompositions , 2000, SIAM J. Matrix Anal. Appl..

[49]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[50]  Pierre Comon,et al.  Nonnegative approximations of nonnegative tensors , 2009, ArXiv.

[51]  S. Leurgans,et al.  A Decomposition for Three-Way Arrays , 1993, SIAM J. Matrix Anal. Appl..

[52]  Dinh Tuan Pham,et al.  Joint Approximate Diagonalization of Positive Definite Hermitian Matrices , 2000, SIAM J. Matrix Anal. Appl..

[53]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[54]  Lieven De Lathauwer,et al.  On the Uniqueness of the Canonical Polyadic Decomposition of Third-Order Tensors - Part I: Basic Results and Uniqueness of One Factor Matrix , 2013, SIAM J. Matrix Anal. Appl..

[55]  F. L. Hitchcock.  The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[56]  C. D. Boor,et al.  Polynomial interpolation in several variables , 1994 .

[57]  Eric Moreau,et al.  Joint Matrices Decompositions and Blind Source Separation , 2014 .

[58]  J. Rhodes.  A concise proof of Kruskal’s theorem on tensor decomposition , 2009, 0901.1796.

[59]  Chris Peterson,et al.  Induction for secant varieties of Segre varieties , 2006, math/0607191.

[60]  A. Causa,et al.  On the maximum rank of a real binary form , 2010, 1006.5127.

[61]  N. Sidiropoulos,et al.  On the uniqueness of multilinear decomposition of N‐way arrays , 2000 .

[62]  Christopher J. Hillar,et al.  Most Tensor Problems Are NP-Hard , 2009, JACM.

[63]  V. Strassen.  Rank and optimal computation of generic tensors , 1983 .

[64]  L. Lathauwer,et al.  On the Best Rank-1 and Rank-( , 2004 .

[65]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[66]  Grazia Lotti,et al.  O(n2.7799) Complexity for n*n Approximate Matrix Multiplication , 1979, Inf. Process. Lett..

[67]  Pierre Comon,et al.  Enhanced Line Search: A Novel Method to Accelerate PARAFAC , 2008, SIAM J. Matrix Anal. Appl..

[68]  B. Reznick.  Sums of Even Powers of Real Linear Forms , 1992 .

[69]  Alessandra Bernardi,et al.  Computing symmetric rank for symmetric tensors , 2009, J. Symb. Comput..

[70]  Pierre Comon,et al.  Tensor Decompositions, State of the Art and Applications , 2002 .

[71]  Pierre Comon,et al.  Symmetric tensor decomposition , 2009, 2009 17th European Signal Processing Conference.

[72]  Volume Assp,et al.  ACOUSTICS. SPEECH. AND SIGNAL PROCESSING , 1983 .

[73]  Klaus Obermayer,et al.  Quadratic optimization for simultaneous matrix diagonalization , 2006, IEEE Transactions on Signal Processing.

[74]  Rasmus Bro,et al.  Improving the speed of multiway algorithms: Part II: Compression , 1998 .

[75]  Pierre Comon,et al.  Tensor decompositions with banded matrix factors , 2013 .

[76]  Florian Roemer,et al.  A semi-algebraic framework for approximate CP decompositions via simultaneous matrix diagonalizations (SECSI) , 2013, Signal Process..

[77]  Pierre Comon,et al.  Computing the polyadic decomposition of nonnegative third order tensors , 2011, Signal Process..

[78]  Bruce Reznick.  LAWS OF INERTIA IN HIGHER DEGREE BINARY FORMS , 2009 .

[79]  Laurent Albera,et al.  Semi-algebraic canonical decomposition of multi-way arrays and Joint Eigenvalue Decomposition , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[80]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[81]  Pierre Comon,et al.  Blind Multilinear Identification , 2012, IEEE Transactions on Information Theory.

[82]  Anthony Iarrobino,et al.  Inverse System of a Symbolic Power, I , 1995 .

[83]  Shmuel Friedland,et al.  The Number of Singular Vector Tuples and Uniqueness of Best Rank-One Approximation of Tensors , 2012, Found. Comput. Math..

[84]  Andrzej Cichocki,et al.  Tensor completion throughmultiple Kronecker product decomposition , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[85]  Rick Miranda,et al.  LINEAR SYSTEMS OF PLANE CURVES , 1999 .

[86]  Tomas Sauer,et al.  Polynomial interpolation in several variables , 2000, Adv. Comput. Math..

[87]  Arie Yeredor,et al.  Non-orthogonal joint diagonalization in the least-squares sense with application in blind source separation , 2002, IEEE Trans. Signal Process..

[88]  Philippe Forster,et al.  Numerical performance of a tensor MUSIC algorithm based on HOSVD for a mixture of polarized sources , 2013, 21st European Signal Processing Conference (EUSIPCO 2013).

[89]  Zbynek Koldovský,et al.  Weight Adjusted Tensor Method for Blind Separation of Underdetermined Mixtures of Nonstationary Sources , 2011, IEEE Transactions on Signal Processing.

[90]  Joos Vandewalle,et al.  Computation of the Canonical Decomposition by Means of a Simultaneous Generalized Schur Decomposition , 2005, SIAM J. Matrix Anal. Appl..

[91]  T. Howell,et al.  Global properties of tensor rank , 1978 .

[92]  P. Comon,et al.  Tensor decompositions, alternating least squares and other tales , 2009 .

[93]  PROCEssIng magazInE.  IEEE Signal Processing Magazine , 2004 .

[94]  Andrzej Cichocki,et al.  CANDECOMP/PARAFAC Decomposition of High-Order Tensors Through Tensor Reshaping , 2012, IEEE Transactions on Signal Processing.

[95]  Nikos D. Sidiropoulos,et al.  Parallel factor analysis in sensor array processing , 2000, IEEE Trans. Signal Process..