Simple Neuron Models for Independent Component Analysis

Recently, several neural algorithms have been introduced for Independent component Analysis. Here we approach the problem from the point of view of a single neuron. First, simple Hebbian-like learning rules are introduced for estimating one of the independent components from sphered data. Some of the learning rules can be used to estimate an independent components which has a negative kurtosis, and the others estimate a component of positive kurtosis. Next, a two-unit system is introduced to estimate an independent component of any kurtosis. The results are then generalized to estimate independent components from non-sphered (raw) mixtures. To separate several independent components, a system of several neurons with linear negative feedback is used. The convergence of the learning rules is rigorously proven without any unnecessary hypotheses on the distribution of the independent components.

[1]  Maurice G. Kendall,et al.  The advanced theory of statistics , 1945 .

[2]  G. Pflug.  Stochastic Approximation Methods for Constrained and Unconstrained Systems - Kushner, HJ.; Clark, D.S. , 1980 .

[3]  D. Donoho.  ON MINIMUM ENTROPY DECONVOLUTION , 1981 .

[4]  E. Oja,et al.  On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix , 1985 .

[5]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[6]  Erkki Oja,et al.  Principal components, minor components, and linear neural networks , 1992, Neural Networks.

[7]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[8]  Nathalie Delfosse,et al.  Adaptive blind separation of independent sources: A deflation approach , 1995, Signal Process..

[9]  Erkki Oja,et al.  Image Feature Extraction Using Independent Component Analysis , 1996 .

[10]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[11]  Zied Malouche,et al.  Extended anti-Hebbian adaptation for unsupervised source extraction , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[12]  Aapo Hyvärinen,et al.  Purely Logical Neural Principal Component and Independent Component Learning , 1996, ICANN.

[13]  A. Hyvärinen,et al.  Nonlinear Blind Source Separation by Self-Organizing Maps , 1996 .

[14]  Erkki Oja,et al.  One-unit Learning Rules for Independent Component Analysis , 1996, NIPS.

[15]  Erkki Oja,et al.  A class of neural networks for independent component analysis , 1997, IEEE Trans. Neural Networks.

[16]  Erkki Oja,et al.  Applications of neural blind separation to signal and image processing , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[17]  A. Hyvarinen,et al.  One-unit contrast functions for independent component analysis: a statistical analysis , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[18]  Erkki Oja,et al.  Independent component analysis by general nonlinear Hebbian-like learning rules , 1998, Signal Process..