Canonical Polyadic Decomposition with a Columnwise Orthonormal Factor Matrix

Canonical polyadic decomposition (CPD) of a higher-order tensor is an important tool in mathematical engineering. In many applications at least one of the matrix factors is constrained to be columnwise orthonormal. We first derive a relaxed condition that guarantees uniqueness of the CPD under this constraint. Second, we give a simple proof of the existence of the optimal low-rank approximation of a tensor in the case that a factor matrix is columnwise orthonormal. Third, we derive numerical algorithms for the computation of the constrained CPD. In particular, orthogonality-constrained versions of the CPD methods based on simultaneous matrix diagonalization and alternating least squares are presented. Numerical experiments are reported.

[1]  Richard A. Harshman,et al.  Foundations of the PARAFAC procedure: Models and conditions for an "explanatory" multi-model factor analysis , 1970 .

[2]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..

[3]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[4]  C. F. Beckmann,et al.  Tensorial extensions of independent component analysis for multisubject FMRI analysis , 2005, NeuroImage.

[5]  A. Stegeman,et al.  On the Non-Existence of Optimal Solutions and the Occurrence of “Degeneracy” in the CANDECOMP/PARAFAC Model , 2008, Psychometrika.

[6]  Nikos D. Sidiropoulos,et al.  Almost-sure identifiability of multidimensional harmonic retrieval , 2001, IEEE Trans. Signal Process..

[7]  David Brie,et al.  DOA estimation for polarized sources on a vector-sensor array by PARAFAC decomposition of the fourth-order covariance tensor , 2008, 2008 16th European Signal Processing Conference.

[8]  Sabine Van Huffel,et al.  Best Low Multilinear Rank Approximation of Higher-Order Tensors, Based on the Riemannian Trust-Region Scheme , 2011, SIAM J. Matrix Anal. Appl..

[9]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[10]  Sabine Van Huffel,et al.  Tucker compression and local optima , 2011 .

[11]  Bart De Moor,et al.  On the blind separation of non-circular sources , 2002, 2002 11th European Signal Processing Conference.

[12]  Alwin Stegeman,et al.  On Uniqueness of the nth Order Tensor Decomposition into Rank-1 Terms with Linear Independence in One Mode , 2010, SIAM J. Matrix Anal. Appl..

[13]  P. Comon Independent Component Analysis , 1992 .

[14]  J. Kruskal Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[15]  J. Chang,et al.  Analysis of individual differences in multidimensional scaling via an n-way generalization of “Eckart-Young” decomposition , 1970 .

[16]  Daniel Graupe,et al.  Topographic component (Parallel Factor) analysis of multichannel evoked potentials: Practical issues in trilinear spatiotemporal decomposition , 2005, Brain Topography.

[17]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[18]  John B. Moore,et al.  Singular Value Decomposition , 1994 .

[19]  R. A. Harshman,et al.  Data preprocessing and the extended PARAFAC model , 1984 .

[20]  Yousef Saad,et al.  On the Tensor SVD and the Optimal Low Rank Orthogonal Approximation of Tensors , 2008, SIAM J. Matrix Anal. Appl..

[21]  Antoine Souloumiac,et al.  Jacobi Angles for Simultaneous Diagonalization , 1996, SIAM J. Matrix Anal. Appl..

[22]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[23]  Nikos D. Sidiropoulos,et al.  Kruskal's permutation lemma and the identification of CANDECOMP/PARAFAC and bilinear models with constant modulus constraints , 2004, IEEE Transactions on Signal Processing.

[24]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[25]  L. De Lathauwer,et al.  Canonical decomposition of even higher order cumulant arrays for blind underdetermined mixture identification , 2008, 2008 5th IEEE Sensor Array and Multichannel Signal Processing Workshop.

[26]  P. Kroonenberg Applied Multiway Data Analysis , 2008 .

[27]  B. Everitt,et al.  Three-Mode Principal Component Analysis. , 1986 .

[28]  H. Kiers Towards a standardized notation and terminology in multiway analysis , 2000 .

[29]  Carla D. Moravitz Martin,et al.  A Jacobi-Type Method for Computing Orthogonal Tensor Decompositions , 2008, SIAM J. Matrix Anal. Appl..

[30]  L. De Lathauwer,et al.  Algebraic methods after prewhitening , 2010 .

[31]  Berkant Savas,et al.  Quasi-Newton Methods on Grassmannians and Multilinear Approximations of Tensors , 2009, SIAM J. Sci. Comput..

[32]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[33]  Lieven De Lathauwer,et al.  Fourth-Order Cumulant-Based Blind Identification of Underdetermined Mixtures , 2007, IEEE Transactions on Signal Processing.

[34]  B. De Moor,et al.  ICA techniques for more sources than sensors , 1999, Proceedings of the IEEE Signal Processing Workshop on Higher-Order Statistics. SPW-HOS '99.

[35]  N. Sidiropoulos,et al.  On the uniqueness of multilinear decomposition of N‐way arrays , 2000 .

[36]  J. Berge,et al.  On uniqueness in candecomp/parafac , 2002 .

[37]  Pierre Comon Independent component analysis - a new concept? signal processing , 1994 .

[38]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[39]  Nikos D. Sidiropoulos,et al.  Blind PARAFAC receivers for DS-CDMA systems , 2000, IEEE Trans. Signal Process..

[40]  L. Lathauwer,et al.  On the best low multilinear rank approximation of higher-order tensors , 2010 .

[41]  Athina P. Petropulu,et al.  Joint singular value decomposition - a new tool for separable representation of images , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[42]  T. Berge Least squares optimization in multivariate analysis , 2005 .

[43]  P. Comon,et al.  Blind Identification of Overcomplete MixturEs of sources (BIOME) , 2004 .

[44]  Luc Deneire,et al.  Parafac with orthogonality in one mode and applications in DS-CDMA systems , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.