Linear and nonlinear ICA based on mutual information - the MISEP method

MISEP is a method for linear and nonlinear ICA, that is able to handle a large variety of situations. It is an extension of the well known INFOMAX method, in two directions: (1) handling of nonlinear mixtures, and (2) learning the nonlinearities to be used at the outputs. The method can therefore separate linear and nonlinear mixtures of components with a wide range of statistical distributions. This paper presents the basis of the MISEP method, as well as experimental results obtained with it. New results show the applicability of the method to mixtures of up to 10 sources, and suggest that its performance scales relatively well with the dimensionality of the problem. The results also show that, although the nonlinear blind source separation problem is ill-posed, the use of regularization allows the problem to be solved when the mixture is not too strongly nonlinear.

[1]  Christian Jutten,et al.  Entropy Optimization - Application to Blind Source Separation , 1997, ICANN.

[2]  J. Cardoso Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[3]  Harri Valpola Nonlinear independent component analysis using ensemble learning: Theory , 2000 .

[4]  Aapo Hyvärinen,et al.  Topographic Independent Component Analysis , 2001, Neural Computation.

[5]  Ali Mansour,et al.  Blind Separation of Sources , 1999 .

[6]  R. A. Redol FASTER TRAINING IN NONLINEAR ICA USING MISEP , 2003 .

[7]  J. Urgen Schmidhuber,et al.  Learning Factorial Codes by Predictability Minimization , 1992 .

[8]  L. B. Almeida,et al.  An Objective Function for Independence , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[9]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[10]  Francesco Palmieri,et al.  Multi-Layer Independent Component Analysis (MLICA) , 1999 .

[11]  Te-Won Lee,et al.  Nonlinear approaches to Independent Component Analysis , 2000 .

[12]  Gustavo Deco,et al.  Nonlinear higher-order statistical decorrelation by volume-conserving neural architectures , 1995, Neural Networks.

[13]  L. B. Almeida,et al.  MISEP-an ICA method for linear and nonlinear mixtures, based on mutual information , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[14]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[15]  Aapo Hyvärinen,et al.  Nonlinear independent component analysis: Existence and uniqueness results , 1999, Neural Networks.

[16]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[17]  Andrzej Cichocki,et al.  Information-theoretic approach to blind separation of sources in non-linear mixture , 1998, Signal Process..

[18]  A. J. Bell THE CO-INFORMATION LATTICE , 2003 .

[19]  L. Almeida,et al.  ICA of linear and nonlinear mixtures based on mutual information , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[20]  Gilles Burel,et al.  Blind separation of sources: A nonlinear neural algorithm , 1992, Neural Networks.

[21]  Dominique Martinez,et al.  Nonlinear blind source separation using kernels , 2003, IEEE Trans. Neural Networks.

[22]  Luís B. Almeida,et al.  MISEP -- Linear and Nonlinear ICA Based on Mutual Information , 2003, J. Mach. Learn. Res..

[23]  HyvärinenAapo,et al.  Nonlinear independent component analysis , 1999 .

[24]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Sub-Gaussian and Super-Gaussian Sources , 1999, Neural Comput..

[25]  G. Darmois,et al.  Analyse générale des liaisons stochastiques: etude particulière de l'analyse factorielle linéaire , 1953 .

[26]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[27]  Terrence J. Sejnowski,et al.  Nonlinear blind source separation using kernel feature spaces , 2001 .