Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources

An extension of the infomax algorithm of Bell and Sejnowski (1995) is presented that is able blindly to separate mixed signals with sub- and supergaussian source distributions. This was achieved by using a simple type of learning rule first derived by Girolami (1997) by choosing negentropy as a projection pursuit index. Parameterized probability distributions that have sub- and supergaussian regimes were used to derive a general learning rule that preserves the simple architecture proposed by Bell and Sejnowski (1995), is optimized using the natural gradient by Amari (1998), and uses the stability analysis of Cardoso and Laheld (1996) to switch between sub- and supergaussian regimes. We demonstrate that the extended infomax algorithm is able to separate 20 sources with a variety of source distributions easily. Applied to high-dimensional data from electroencephalographic recordings, it is effective at separating artifacts such as eye blinks and line noise from weaker electrical signals that arise from sources in the brain.

[1]  K. Pearson.  Contributions to the Mathematical Theory of Evolution , 1894 .

[2]  M. Kendall,et al.  Kendall's Advanced Theory of Statistics: Volume 1 Distribution Theory , 1987 .

[3]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[4]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[5]  P. Berg,et al.  Dipole models of eye movements and blinks. , 1991, Electroencephalography and clinical neurophysiology.

[6]  J. Cardoso,et al.  Blind beamforming for non-gaussian signals , 1993 .

[7]  J. Nadal.  Non linear neurons in the low noise limit : a factorial code maximizes information transferJean , 1994 .

[8]  J. Nadal,et al.  Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer Network 5 , 1994 .

[9]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[10]  C. D. Kemp,et al.  Kendall's Advanced Theory of Statistics, Vol. 1: Distribution Theory. , 1995 .

[11]  Andrzej Cichocki,et al.  Robust learning algorithm for blind separation of signals , 1994 .

[12]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[13]  Tzyy-Ping Jung,et al.  Independent Component Analysis of Electroencephalographic Data , 1995, NIPS.

[14]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[15]  G. Deco,et al.  An Information-Theoretic Approach to Neural Computing , 1997, Perspectives in Neural Computing.

[16]  Yoram Baram,et al.  Multidimensional density shaping by sigmoids , 1996, IEEE Trans. Neural Networks.

[17]  Olivier Herbeaux,et al.  A A unifying framework , 1996 .

[18]  Barak A. Pearlmutter,et al.  A Context-Sensitive Generalization of ICA , 1996 .

[19]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[20]  Juha Karhunen,et al.  Neural approaches to independent component analysis and source separation , 1996, ESANN.

[21]  Aapo Hyvärinen,et al.  A Fast Fixed-Point Algorithm for Independent Component Analysis , 1997, Neural Computation.

[22]  Shun-ichi Amari,et al.  Blind source separation-semiparametric statistical approach , 1997, IEEE Trans. Signal Process..

[23]  Erkki Oja,et al.  The nonlinear PCA learning rule in independent component analysis , 1997, Neurocomputing.

[24]  H. H. Yang,et al.  Information backpropagation for blind separation of sources in nonlinear mixture , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[25]  C. Fyfe,et al.  Generalised independent component analysis through unsupervised learning with emergent Bussgang properties , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[26]  Erkki Oja,et al.  A class of neural networks for independent component analysis , 1997, IEEE Trans. Neural Networks.

[27]  Juha Karhunen,et al.  A Maximum Likelihood Approach to Nonlinear Blind Source Separation , 1997, ICANN.

[28]  H. Yangy,et al.  Information Back-propagation for Blind Separation of Sources in Non-linear Mixture , 1997 .

[29]  Tzyy-Ping Jung,et al.  Extended ICA Removes Artifacts from Electroencephalographic Recordings , 1997, NIPS.

[30]  D. Chakrabarti,et al.  A fast fixed - point algorithm for independent component analysis , 1997 .

[31]  Jack D. Cowan,et al.  Faithful Representation Of Separable Input Distributions , 1997, Neural Computation.

[32]  Christian Jutten,et al.  Nonlinear source separation: the post-nonlinear mixtures , 1997, ESANN.

[33]  Eric Moulines,et al.  Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[34]  S Makeig,et al.  Blind separation of auditory event-related brain responses into independent components. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[35]  Shun-ichi Amari,et al.  Stability Analysis Of Adaptive Blind Source Separation , 1997 .

[36]  J. Cardoso.  Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[37]  Juan K. Lin,et al.  Faithful Representation of Separable Distributions , 1997, Neural Computation.

[38]  Andrzej Cichocki,et al.  Stability Analysis of Learning Algorithms for Blind Source Separation , 1997, Neural Networks.

[39]  Te-Won Lee,et al.  Blind source separation of nonlinear mixing models , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[40]  Terrence J. Sejnowski,et al.  Learning Nonlinear Overcomplete Representations for Efficient Coding , 1997, NIPS.

[41]  Mark Girolami,et al.  Extraction of independent signal sources using a deflationary exploratory projection pursuit network , 1997 .

[42]  Néstor Parga,et al.  Redundancy Reduction and Independent Component Analysis: Conditions on Cumulants and Adaptive Approaches , 1997, Neural Computation.

[43]  Philippe Garat,et al.  Blind separation of mixture of independent sources through a quasi-maximum likelihood approach , 1997, IEEE Trans. Signal Process..

[44]  Te-Won Lee,et al.  Independent Component Analysis , 1998, Springer US.

[45]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[46]  T. Ens,et al.  Blind signal separation : statistical principles , 1998 .

[47]  S Makeig,et al.  Spatially independent activity patterns in functional MRI data during the stroop color-naming task. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[48]  Terrence J. Sejnowski,et al.  Unsupervised Classification with Non-Gaussian Mixture Models Using ICA , 1998, NIPS.

[49]  Mark A. Girolami,et al.  An Alternative Perspective on Adaptive Independent Component Analysis Algorithms , 1998, Neural Computation.

[50]  M. Laubach,et al.  Independent component analyses for quantifying neuronal ensemble interactions , 1999, Journal of Neuroscience Methods.

[51]  A. J. Bell,et al.  A Unifying Information-Theoretic Framework for Independent Component Analysis , 2000 .

[52]  Richard M. Everson,et al.  Independent Component Analysis: A Flexible Nonlinearity and Decorrelating Manifold Approach , 1999, Neural Computation.

[53]  Jürgen Schmidhuber,et al.  Feature Extraction Through LOCOCODE , 1999, Neural Computation.

[54]  Terrence J. Sejnowski,et al.  Blind source separation of more sources than mixtures using overcomplete representations , 1999, IEEE Signal Processing Letters.

[55]  T. Sejnowski,et al.  Removal of eye activity artifacts from visual event-related potentials in normal and clinical subjects , 2000, Clinical Neurophysiology.

[56]  Terrence J. Sejnowski,et al.  ICA Mixture Models for Unsupervised Classification of Non-Gaussian Classes and Automatic Context Switching in Blind Signal Separation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  M. McKeown.  Detection of Consistently Task-Related Activations in fMRI Data with Hybrid Independent Component Analysis , 2000, NeuroImage.

[58]  Thomas P. von Hoff,et al.  Transpose properties in the stability and performance of the classic adaptive algorithms for blind source separation and deconvolution , 2000, Signal Process..

[59]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[60]  Nikos Fakotakis,et al.  Independent component analysis applied to feature extraction for robust automatic speech recognition , 2000 .

[61]  L. Finkel,et al.  Color-opponent receptive fields derived from independent component analysis of natural images , 2000, Vision Research.

[62]  T. W. Lee,et al.  Chromatic structure of natural scenes. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[63]  Robert Hecht-Nielsen,et al.  A Biologically Motivated Solution to the Cocktail Party Problem , 2001, Neural Computation.

[64]  Max Welling,et al.  A Constrained EM Algorithm for Independent Component Analysis , 2001, Neural Computation.

[65]  Nikos A. Vlassis,et al.  Efficient source adaptivity in independent component analysis , 2001, IEEE Trans. Neural Networks.

[66]  Thomas P. von Hoff,et al.  Blind separation of signals with mixed kurtosis signs using threshold activation functions , 2001, IEEE Trans. Neural Networks.

[67]  T. Sejnowski,et al.  Analysis and visualization of single‐trial event‐related potentials , 2001, Human brain mapping.

[68]  Tzyy-Ping Jung,et al.  Imaging brain dynamics using independent component analysis , 2001, Proc. IEEE.

[69]  S. Amari.  Natural Gradient Works Eciently in Learning , 2022 .