Joint Independent Subspace Analysis: Uniqueness and Identifiability

This paper deals with the identifiability of joint independent subspace analysis (JISA). JISA is a recently-proposed framework that subsumes independent vector analysis (IVA) and independent subspace analysis (ISA). Each underlying mixture can be regarded as a dataset; therefore, JISA can be used for data fusion. In this paper, we assume that each dataset is an overdetermined mixture of several multivariate Gaussian processes, each of which has independent and identically distributed samples. This setup is not identifiable when each mixture is considered individually. Given these assumptions, JISA can be restated as coupled block diagonalization (CBD) of its correlation matrices. Hence, JISA identifiability is tantamount to CBD uniqueness. In this work, we provide necessary and sufficient conditions for uniqueness and identifiability of JISA and CBD. Our analysis is based on characterizing all the cases in which the Fisher information matrix is singular. We prove that non-identifiability may occur only due to pairs of underlying random processes with the same dimension. Our results provide further evidence that irreducibility has a central role in the uniqueness analysis of block-based decompositions. Our contribution extends previous results on the uniqueness and identifiability of ISA, IVA, coupled matrix and tensor decompositions. We provide examples to illustrate our results.

[1]  Rasmus Bro,et al.  MULTI-WAY ANALYSIS IN THE FOOD INDUSTRY Models, Algorithms & Applications , 1998 .

[2]  Christian Jutten,et al.  A Generalization to Schur's Lemma with an Application to Joint Independent Subspace Analysis , 2016 .

[3]  J. Kruskal.  Three-way arrays: rank and uniqueness of trilinear decompositions, with application to arithmetic complexity and statistics , 1977 .

[4]  Shuangzhe Liu,et al.  Hadamard, Khatri-Rao, Kronecker and Other Matrix Products , 2008 .

[5]  J. Magnus,et al.  The Commutation Matrix: Some Properties and Applications , 1979 .

[6]  Jean-François Cardoso,et al.  Component Separation With Flexible Models—Application to Multichannel Astrophysical Observations , 2008, IEEE Journal of Selected Topics in Signal Processing.

[7]  Vince D. Calhoun,et al.  Multidataset independent subspace analysis extends independent vector analysis , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[8]  Adem Kiliçman,et al.  Some new connections between matrix products for partitioned and non-partitioned matrices , 2007, Comput. Math. Appl..

[9]  R. Liu,et al.  AMUSE: a new blind identification algorithm , 1990, IEEE International Symposium on Circuits and Systems.

[10]  Yoshihiro Kanno,et al.  A numerical algorithm for block-diagonal decomposition of matrix $${*}$$-algebras with application to semidefinite programming , 2010 .

[11]  Christian Jutten,et al.  Joint Independent Subspace Analysis Using Second-Order Statistics , 2016, IEEE Transactions on Signal Processing.

[12]  Christian Jutten,et al.  Joint Independent Subspace Analysis by Coupled Block Decomposition: Non-Identifiable Cases , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[13]  Hagit Messer,et al.  Identifiability of second-order multidimensional ICA , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[14]  Vince D. Calhoun,et al.  Joint Blind Source Separation by Multiset Canonical Correlation Analysis , 2009, IEEE Transactions on Signal Processing.

[15]  T. W. Anderson,et al.  An Introduction to Multivariate Statistical Analysis , 1959 .

[16]  Rasmus Bro,et al.  Forecasting Chronic Diseases Using Data Fusion. , 2017, Journal of proteome research.

[17]  Hagit Messer,et al.  Second-Order Multidimensional ICA: Performance Analysis , 2012, IEEE Transactions on Signal Processing.

[18]  Pierre Comon,et al.  Handbook of Blind Source Separation: Independent Component Analysis and Applications , 2010 .

[19]  Tülay Adali,et al.  Joint blind source separation from second-order statistics: Necessary and sufficient identifiability conditions , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  Christian Jutten,et al.  Joint analysis of multiple datasets by cross-cumulant tensor (block) diagonalization , 2016, 2016 IEEE Sensor Array and Multichannel Signal Processing Workshop (SAM).

[21]  Hagit Messer,et al.  Blind Separation of Multi-Dimensional Components via Subspace Decomposition: Performance Analysis , 2013, IEEE Transactions on Signal Processing.

[22]  A. Belouchrani,et al.  Blind separation of non stationary sources using joint block diagonalization , 2001, Proceedings of the 11th IEEE Signal Processing Workshop on Statistical Signal Processing (Cat. No.01TH8563).

[23]  Christian Jutten,et al.  Joint blind source separation of multidimensional components: Model and algorithm , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[24]  Christian Jutten,et al.  Schur's Lemma for Coupled Reducibility and Coupled Normality , 2018, SIAM J. Matrix Anal. Appl..

[25]  Te-Won Lee,et al.  Independent Vector Analysis: An Extension of ICA to Multivariate Components , 2006, ICA.

[26]  Tülay Adali,et al.  Joint blind source separation by generalized joint diagonalization of cumulant matrices , 2011, Signal Process..

[27]  Alan J. Mayne,et al.  Generalized Inverse of Matrices and its Applications , 1972 .

[28]  Wim Van Paesschen,et al.  Block term decomposition for modelling epileptic seizures , 2014, EURASIP J. Adv. Signal Process..

[29]  Anja Vogler,et al.  An Introduction to Multivariate Statistical Analysis , 2004 .

[30]  Jean-François Cardoso,et al.  Multidimensional independent component analysis , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[31]  Harold W. Gutch,et al.  Second Order Subspace Analysis and Simple Decompositions , 2010, LVA/ICA.

[32]  Harry L. Van Trees,et al.  Optimum Array Processing: Part IV of Detection, Estimation, and Modulation Theory , 2002 .

[33]  L. Lathauwer,et al.  Fetal electrocardiogram extraction by source subspace separation , 1995 .

[34]  Yingbo Hua,et al.  Fast maximum likelihood for blind identification of multiple FIR channels , 1996, IEEE Trans. Signal Process..

[35]  Michael A. Casey,et al.  Separation of Mixed Audio Sources By Independent Subspace Analysis , 2000, ICMC.

[36]  Jarkko Ylipaavalniemi,et al.  Subspaces of Spatially Varying Independent Components in fMRI , 2007, ICA.

[37]  Tülay Adalı,et al.  Diversity in Independent Component and Vector Analyses: Identifiability, algorithms, and applications in medical imaging , 2014, IEEE Signal Processing Magazine.

[38]  Lieven De Lathauwer,et al.  Coupled Canonical Polyadic Decompositions and (Coupled) Decompositions in Multilinear Rank-(Lr, n, Lr, n, 1) Terms - Part I: Uniqueness , 2015, SIAM J. Matrix Anal. Appl..

[39]  Christian Jutten,et al.  Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects , 2015, Proceedings of the IEEE.

[40]  Ronald Phlypo,et al.  Independent Vector Analysis: Identification Conditions and Performance Bounds , 2013, IEEE Transactions on Signal Processing.

[41]  Vince D. Calhoun,et al.  Flexible large-scale fMRI analysis: A survey , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[42]  Joe Brewer,et al.  Kronecker products and matrix calculus in system theory , 1978 .

[43]  T. Rothenberg.  Identification in Parametric Models , 1971 .

[44]  Qiu-Hua Lin,et al.  Double Coupled Canonical Polyadic Decomposition for Joint Blind Source Separation , 2016, IEEE Transactions on Signal Processing.

[45]  J Fontecave-Jallon,et al.  Non-invasive fetal monitoring using electrocardiography and phonocardiography: A preliminary study. , 2018, Journal of gynecology obstetrics and human reproduction.

[46]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[47]  Christian Jutten,et al.  An alternative proof for the identifiability of independent vector analysis using second order statistics , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[48]  T. Ens,et al.  Blind signal separation : statistical principles , 1998 .

[49]  Harold W. Gutch,et al.  Uniqueness of linear factorizations into independent subspaces , 2012, J. Multivar. Anal..

[50]  Christian Jutten,et al.  A New Link Between Joint Blind Source Separation Using Second Order Statistics and the Canonical Polyadic Decomposition , 2018, LVA/ICA.

[51]  Christian Jutten,et al.  Joint Independent Subspace Analysis: A Quasi-Newton Algorithm , 2015, LVA/ICA.

[52]  R. Horn,et al.  Block-matrix generalizations of Schur's basic theorems on Hadamard products , 1992 .

[53]  Jean-Francois Cardoso,et al.  Foreground component separation with generalised ILC , 2011, 1103.1166.

[54]  Tülay Adali,et al.  A maximum likelihood approach for independent vector analysis of Gaussian data sets , 2011, 2011 IEEE International Workshop on Machine Learning for Signal Processing.

[55]  I. Schur,et al.  Neue Begründung der Theorie der Gruppencharaktere: Sitzungsberichte der Preussischen Akademie der Wissenschaften 1905, Physikalisch-Mathematische Klasse, 406 – 432 , 1973 .

[56]  Alexander Graham,et al.  Kronecker Products and Matrix Calculus: With Applications , 1981 .

[57]  Vince D. Calhoun,et al.  Quantifying the Interaction and Contribution of Multiple Datasets in Fusion: Application to the Detection of Schizophrenia , 2017, IEEE Transactions on Medical Imaging.

[58]  R. P. McDonald,et al.  A simple comprehensive model for the analysis of covariance structures: Some remarks on applications , 1980 .

[59]  Lieven De Lathauwer,et al.  Decompositions of a Higher-Order Tensor in Block Terms - Part II: Definitions and Uniqueness , 2008, SIAM J. Matrix Anal. Appl..