Subtracting a best rank-1 approximation may increase tensor rank

Is has been shown that a best rank-R approximation of an order-k tensor may not exist when R ≥ 2 and k ≥ 3. This poses a serious problem to data analysts using Candecomp/Parafac and related models. It has been observed numerically that, generally, this issue cannot be solved by consecutively computing and substracting best rank-1 approximations. The reason for this is that subtracting a best rank-1 approximation generally does not decrease tensor rank. In this paper, we provide a mathematical treatment of this property for real-valued 2 × 2 × 2 tensors, with symmetric tensors as a special case. Regardless of the symmetry, we show that for generic 2 × 2 × 2 tensors (which have rank 2 or 3), subtracting a best rank-1 approximation will result in a tensor that has rank 3 and lies on the boundary between the rank-2 and rank-3 sets. Hence, for a typical tensor of rank 2, subtracting a best rank-1 approximation has increased the tensor rank.

[1]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[2]  Yiju Wang,et al.  On the best rank-1 approximation to higher-order symmetric tensors , 2007, Math. Comput. Model..

[3]  J. Astola,et al.  Mathematics in signal processing , 1989 .

[4]  A. Stegeman.  Degeneracy in Candecomp/Parafac and Indscal Explained For Several Three-Sliced Arrays With A Two-Valued Typical Rank , 2007, Psychometrika.

[5]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[6]  Gene H. Golub,et al.  Rank-One Approximation to High Order Tensors , 2001, SIAM J. Matrix Anal. Appl..

[7]  Michael Clausen,et al.  Algebraic complexity theory , 1997, Grundlehren der mathematischen Wissenschaften.

[8]  Phillip A. Regalia,et al.  On the Best Rank-1 Approximation of Higher-Order Supersymmetric Tensors , 2001, SIAM J. Matrix Anal. Appl..

[9]  Pierre Comon.  Independent component analysis - a new concept? signal processing , 1994 .

[10]  V. Strassen.  Rank and optimal computation of generic tensors , 1983 .

[11]  L. Lathauwer,et al.  On the Best Rank-1 and Rank-( , 2004 .

[12]  J. Leeuw,et al.  Explicit candecomp/parafac solutions for a contrived 2 × 2 × 2 array of rank three , 1988 .

[13]  Pierre Comon,et al.  Decomposition of quantics in sums of powers of linear forms , 1996, Signal Process..

[14]  Lieven De Lathauwer,et al.  A Method to Avoid Diverging Components in the Candecomp/Parafac Model for Generic I˟J˟2 Arrays , 2008, SIAM J. Matrix Anal. Appl..

[15]  Stavros M. Panas,et al.  Higher‐Order Statistics , 2006 .

[16]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[17]  Gene H. Golub,et al.  Symmetric Tensors and Symmetric Tensor Rank , 2008, SIAM J. Matrix Anal. Appl..

[18]  E. Oja,et al.  Independent Component Analysis , 2013 .

[19]  A. Stegeman,et al.  On Kruskal's uniqueness condition for the Candecomp/Parafac decomposition , 2007 .

[20]  Pierre Comon,et al.  Blind channel identification and extraction of more sources than sensors , 1998, Optics & Photonics.

[21]  Nikos D. Sidiropoulos,et al.  Kruskal's permutation lemma and the identification of CANDECOMP/PARAFAC and bilinear models with constant modulus constraints , 2004, IEEE Transactions on Signal Processing.

[22]  Alwin Stegeman,et al.  Low-Rank Approximation of Generic p˟q˟2 Arrays and Diverging Components in the Candecomp/Parafac Model , 2008, SIAM J. Matrix Anal. Appl..

[23]  A. Agresti,et al.  Multiway Data Analysis , 1989 .

[24]  F. L. Hitchcock.  Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[25]  Rasmus Bro,et al.  Multi-way Analysis with Applications in the Chemical Sciences , 2004 .

[26]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[27]  Andrew G. Glen,et al.  APPL , 2001 .

[28]  Joseph JáJá.  Optimal Evaluation of Pairs of Bilinear Forms , 1979, SIAM J. Comput..

[29]  T. Ens,et al.  Blind signal separation : statistical principles , 1998 .

[30]  Nikos D. Sidiropoulos,et al.  Parallel factor analysis in sensor array processing , 2000, IEEE Trans. Signal Process..

[31]  Nikos D. Sidiropoulos,et al.  Blind PARAFAC receivers for DS-CDMA systems , 2000, IEEE Trans. Signal Process..

[32]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[33]  J. Berge,et al.  Simplicity of core arrays in three-way principal component analysis and the typical rank of p×q×2 arrays , 1999 .

[34]  P. Kroonenberg.  Applied Multiway Data Analysis , 2008 .

[35]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[36]  G. Rota,et al.  The invariant theory of binary forms , 1984 .

[37]  J. Vandewalle,et al.  An introduction to independent component analysis , 2000 .

[38]  J. Chisholm.  Approximation by Sequences of Padé Approximants in Regions of Meromorphy , 1966 .

[39]  Lieven De Lathauwer,et al.  Tensor-based techniques for the blind separation of DS-CDMA signals , 2007, Signal Process..

[40]  Jean-Francois Cardoso,et al.  Blind signal separation: statistical principles , 1998, Proc. IEEE.

[41]  A. Stegeman,et al.  On the Non-Existence of Optimal Solutions and the Occurrence of “Degeneracy” in the CANDECOMP/PARAFAC Model , 2008, Psychometrika.

[42]  André Lima Férrer de Almeida,et al.  PARAFAC-based unified tensor modeling for wireless communication systems with application to blind multiuser equalization , 2007, Signal Process..

[43]  Volker Strassen,et al.  Algebraic Complexity Theory , 1991, Handbook of Theoretical Computer Science, Volume A: Algorithms and Complexity.

[44]  Liqun Qi,et al.  Rank and eigenvalues of a supersymmetric tensor, the multivariate homogeneous polynomial and the algebraic hypersurface it defines , 2006, J. Symb. Comput..

[45]  Andrzej Cichocki,et al.  Adaptive blind signal and image processing , 2002 .

[46]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[47]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[48]  F. L. Hitchcock.  The Expression of a Tensor or a Polyadic as a Sum of Products , 1927 .

[49]  Mike E. Davies,et al.  Tensor Decompositions, State of the Art and Applications , 2009, 0905.0454.

[50]  Pierre Comon,et al.  Blind identification of under-determined mixtures based on the characteristic function , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[51]  J. Kruskal.  Rank, decomposition, and uniqueness for 3-way and n -way arrays , 1989 .

[52]  J. Berge,et al.  Typical rank and indscal dimensionality for symmetric three-way arrays of order I×2×2 or I×3×3 , 2004 .

[53]  A. Stegeman.  Degeneracy in Candecomp/Parafac explained for p × p × 2 arrays of rank p + 1 or higher , 2006 .

[54]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[55]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..