Fast and unique Tucker decompositions via multiway blind source separation

A multiway blind source separation (MBSS) method is developed to decompose large-scale tensor (multiway array) data. Benefitting from all kinds of well-established constrained low-rank matrix factorization methods, MBSS is quite flexible and able to extract unique and interpretable components with physical meaning. The multilinear structure of Tucker and the essential uniqueness of BSS methods allow MBSS to estimate each component matrix separately from an unfolding matrix in each mode. Consequently, alternating least squares (ALS) iterations, which are considered as the workhorse for tensor decompositions, can be avoided and various robust and efficient dimensionality reduction methods can be easily incorporated to pre-process the data, which makes MBSS extremely fast, especially for large-scale problems. Identification and uniqueness conditions are also discussed. Two practical issues dimensionality reduction and estimation of number of components are also addressed based on sparse and random fibers sampling. Extensive simulations confirmed the validity, flexibility, and high efficiency of the proposed method. We also demonstrated by simulations that the MBSS approach can successfully extract desired components while most existing algorithms may fail for ill-conditioned and large-scale problems.

[1]  Eric Moulines,et al.  A blind source separation technique using second-order statistics , 1997, IEEE Trans. Signal Process..

[2]  P. Tichavský,et al.  Efficient variant of algorithm fastica for independent component analysis attaining the cramer-RAO lower bound , 2005, IEEE/SP 13th Workshop on Statistical Signal Processing, 2005.

[3]  William S Rayens,et al.  Structure-seeking multilinear methods for the analysis of fMRI data , 2004, NeuroImage.

[4]  Andrzej Cichocki,et al.  Adaptive blind signal and image processing , 2002 .

[5]  Fumikazu Miwakeichi,et al.  Decomposing EEG Data into Space-Time-Frequency Components Using Parallel Factor Analysis and Its Relation with Cerebral Blood Flow , 2007, ICONIP.

[6]  E. Oja,et al.  Independent Component Analysis , 2001 .

[7]  J. Wolpaw,et al.  Towards an independent brain–computer interface using steady state visual evoked potentials , 2008, Clinical Neurophysiology.

[8]  Andrzej Cichocki,et al.  Fast Nonnegative Matrix/Tensor Factorization Based on Low-Rank Approximation , 2012, IEEE Transactions on Signal Processing.

[9]  Fabian J. Theis,et al.  Sparse component analysis and blind source separation of underdetermined mixtures , 2005, IEEE Transactions on Neural Networks.

[10]  Vin de Silva,et al.  Tensor rank and the ill-posedness of the best low-rank approximation problem , 2006, math/0607647.

[11]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[12]  Fumikazu Miwakeichi,et al.  Decomposing EEG data into space–time–frequency components using Parallel Factor Analysis , 2004, NeuroImage.

[13]  Laurent Albera,et al.  Iterative methods for the canonical decomposition of multi-way arrays: Application to blind underdetermined mixture identification , 2011, Signal Process..

[14]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[15]  H. Kiers,et al.  Selecting among three-mode principal component models of different types and complexities: a numerical convex hull based method. , 2006, The British journal of mathematical and statistical psychology.

[16]  A. Cichocki,et al.  Generalizing the column–row matrix decomposition to multi-way arrays , 2010 .

[17]  G. Sapiro,et al.  A collaborative framework for 3D alignment and classification of heterogeneous subvolumes in cryo-electron tomography. , 2013, Journal of structural biology.

[18]  Zhinan Zhang,et al.  The rank of a random matrix , 2007, Appl. Math. Comput..

[19]  Joos Vandewalle,et al.  A Multilinear Singular Value Decomposition , 2000, SIAM J. Matrix Anal. Appl..

[20]  Fumikazu Miwakeichi,et al.  Concurrent EEG/fMRI analysis by multiway Partial Least Squares , 2004, NeuroImage.

[21]  Victor Solo,et al.  Dimension Estimation in Noisy PCA With SURE and Random Matrix Theory , 2008, IEEE Transactions on Signal Processing.

[22]  Sabine Van Huffel,et al.  A Combination of Parallel Factor and Independent Component Analysis , 2022 .

[23]  Kyuwan Choi,et al.  Detecting the Number of Clusters in n-Way Probabilistic Clustering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Rasmus Bro,et al.  The N-way Toolbox for MATLAB , 2000 .

[25]  Siep Weiland,et al.  Singular Value Decompositions and Low Rank Approximations of Tensors , 2010, IEEE Transactions on Signal Processing.

[26]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[27]  Sabine Van Huffel,et al.  Algorithm for imposing SOBI-type constraints on the CP model , 2008, 2008 IEEE International Symposium on Circuits and Systems.

[28]  Ian T. Jolliffe,et al.  Independent Component Analysis for Three-Way Data With an Application From Atmospheric Science , 2011 .

[29]  Xingyu Wang,et al.  Multiway Canonical Correlation Analysis for Frequency Components Recognition in SSVEP-Based BCIs , 2011, ICONIP.

[30]  Demetri Terzopoulos,et al.  Multilinear independent components analysis , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[31]  Patrick Dupont,et al.  Canonical decomposition of ictal scalp EEG reliably detects the seizure onset zone , 2007, NeuroImage.

[32]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[33]  Yi Ma,et al.  The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices , 2010, Journal of structural biology.

[34]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[35]  Rasmus Bro,et al.  Multiway analysis of epilepsy tensors , 2007, ISMB/ECCB.

[36]  A. Cichocki,et al.  Tensor decompositions for feature extraction and classification of high dimensional datasets , 2010 .

[37]  Erkki Oja,et al.  Efficient Variant of Algorithm FastICA for Independent Component Analysis Attaining the CramÉr-Rao Lower Bound , 2006, IEEE Transactions on Neural Networks.

[38]  Yi Yang,et al.  Image Clustering Using Local Discriminant Models and Global Integration , 2010, IEEE Transactions on Image Processing.

[39]  Petros Drineas,et al.  Fast Monte Carlo Algorithms for Matrices II: Computing a Low-Rank Approximation to a Matrix , 2006, SIAM J. Comput..

[40]  C. F. Beckmann,et al.  Tensorial extensions of independent component analysis for multisubject FMRI analysis , 2005, NeuroImage.

[41]  Andrzej Cichocki,et al.  Damped Newton Iterations for Nonnegative Matrix Factorization , 2010, Aust. J. Intell. Inf. Process. Syst..

[42]  L. Tucker,et al.  Some mathematical notes on three-mode factor analysis , 1966, Psychometrika.

[43]  Andrzej Cichocki,et al.  Extended HALS algorithm for nonnegative Tucker decomposition and its applications for multiway analysis and classification , 2011, Neurocomputing.

[44]  A. Cichocki Generalized Component Analysis and Blind Source Separation Methods for Analyzing Multichannel Brain Signals , 2006 .

[45]  Dezhong Yao,et al.  Frequency detection with stability coefficient for steady-state visual evoked potential (SSVEP)-based BCIs. , 2008, Journal of neural engineering.

[46]  Lieven De Lathauwer,et al.  A survey of tensor methods , 2009, 2009 IEEE International Symposium on Circuits and Systems.

[47]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[48]  Dave R M Langers,et al.  Unbiased group‐level statistical assessment of independent component maps by means of automated retrospective matching , 2009, Human brain mapping.

[49]  Henk A L Kiers,et al.  A fast method for choosing the numbers of components in Tucker3 analysis. , 2003, The British journal of mathematical and statistical psychology.

[50]  F. Fisher,et al.  The Identification Problem in Econometrics. , 1967 .

[51]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[52]  Xiaorong Gao,et al.  A BCI-based environmental controller for the motion-disabled , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[53]  Andrzej Cichocki,et al.  Canonical Polyadic Decomposition Based on a Single Mode Blind Source Separation , 2012, IEEE Signal Processing Letters.

[54]  Rasmus Bro,et al.  MULTI-WAY ANALYSIS IN THE FOOD INDUSTRY Models, Algorithms & Applications , 1998 .

[55]  Joos Vandewalle,et al.  On the Best Rank-1 and Rank-(R1 , R2, ... , RN) Approximation of Higher-Order Tensors , 2000, SIAM J. Matrix Anal. Appl..

[56]  F. L. Hitchcock Multiple Invariants and Generalized Rank of a P‐Way Matrix or Tensor , 1928 .

[57]  Lars Kai Hansen,et al.  Algorithms for Sparse Nonnegative Tucker Decompositions , 2008, Neural Computation.

[58]  Barak A. Pearlmutter,et al.  Blind Source Separation by Sparse Decomposition in a Signal Dictionary , 2001, Neural Computation.

[59]  Petros Drineas,et al.  CUR matrix decompositions for improved data analysis , 2009, Proceedings of the National Academy of Sciences.

[60]  Lieven De Lathauwer,et al.  A Link between the Canonical Decomposition in Multilinear Algebra and Simultaneous Matrix Diagonalization , 2006, SIAM J. Matrix Anal. Appl..