MISEP -- Linear and Nonlinear ICA Based on Mutual Information

Linear Independent Components Analysis (ICA) has become an important signal processing and data analysis technique, the typical application being blind source separation in a wide range of signals, such as biomedical, acoustical and astrophysical ones. Nonlinear ICA is less developed, but has the potential to become at least as powerful.This paper presents MISEP, an ICA technique for linear and nonlinear mixtures, which is based on the minimization of the mutual information of the estimated components. MISEP is a generalization of the popular INFOMAX technique, which is extended in two ways: (1) to deal with nonlinear mixtures, and (2) to be able to adapt to the actual statistical distributions of the sources, by dynamically estimating the nonlinearities to be used at the outputs. The resulting MISEP method optimizes a network with a specialized architecture, with a single objective function: the output entropy.The paper also briefly discusses the issue of nonlinear source separation. Examples of linear and nonlinear source separation performed by MISEP are presented.

[1]  J. Cardoso Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[2]  Dominique Martinez,et al.  Nonlinear blind source separation using kernels , 2003, IEEE Trans. Neural Networks.

[3]  Andrzej Cichocki,et al.  Information-theoretic approach to blind separation of sources in non-linear mixture , 1998, Signal Process..

[4]  R. Alves Redol,et al.  MISEP - Linear and Nonlinear ICA Based on Mutual Information , 2003 .

[5]  Pierre Comon Independent component analysis - a new concept? signal processing , 1994 .

[6]  Gustavo Deco,et al.  Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures , 1995, Neural Networks.

[7]  Te-Won Lee,et al.  Nonlinear approaches to Independent Component Analysis , 2000 .

[8]  G. Darmois,et al.  Analyse générale des liaisons stochastiques: etude particulière de l'analyse factorielle linéaire , 1953 .

[9]  Gilles Burel,et al.  Blind separation of sources: A nonlinear neural algorithm , 1992, Neural Networks.

[10]  Francesco Palmieri,et al.  Multi-Layer Independent Component Analysis (MLICA) , 1999 .

[11]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[12]  L. Almeida,et al.  ICA of linear and nonlinear mixtures based on mutual information , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[13]  A. J. Bell,et al.  A Unifying Information-Theoretic Framework for Independent Component Analysis , 2000 .

[14]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[15]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[16]  Jürgen Schmidhuber,et al.  Learning Factorial Codes by Predictability Minimization , 1992, Neural Computation.

[17]  Luís B. Almeida,et al.  Linear and nonlinear ICA based on mutual information - the MISEP method , 2004, Signal Process..

[18]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis , 1997, Neural Computation.

[19]  Harri Valpola Nonlinear independent component analysis using ensemble learning: Theory , 2000 .

[20]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[21]  L. B. Almeida,et al.  An Objective Function for Independence , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[22]  Terrence J. Sejnowski,et al.  Nonlinear blind source separation using kernel feature spaces , 2001 .

[23]  Christian Jutten,et al.  Entropy Optimization - Application to Blind Source Separation , 1997, ICANN.

[24]  Antoine Souloumiac,et al.  Jacobi Angles for Simultaneous Diagonalization , 1996, SIAM J. Matrix Anal. Appl..

[25]  L. B. Almeida,et al.  MISEP-an ICA method for linear and nonlinear mixtures, based on mutual information , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[26]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[27]  Luis B. Almeida,et al.  Faster Training in Nonlinear ICA using MISEP , 2002 .

[28]  Aapo Hyvärinen,et al.  Nonlinear independent component analysis: Existence and uniqueness results , 1999, Neural Networks.