Superefficiency in blind source separation

Blind source separation is the problem of extracting independent signals from their mixtures without knowing the mixing coefficients nor the probability distributions of source signals and may be applied to EEG and MEG imaging of the brain. It is already known that certain algorithms work well for the extraction of independent components. The present paper is concerned with superefficiency of these based on the statistical and dynamical analysis. In a statistical estimation using t examples, the covariance of any two extracted independent signals converges to 0 of the order of 1/t. On-line dynamics shows that the covariance is of the order of /spl eta/ when the learning rate /spl eta/ is fixed to a small constant. In contrast with the above general properties, a surprising superefficiency holds in blind source separation under certain conditions where superefficiency implies that covariance decreases in the order of 1/t/sup 2/ or of /spl eta//sup 2/. The paper uses the natural gradient learning algorithm and method of estimating functions to obtain superefficient procedures for both batch estimation and on-line learning. A standardized estimating function is introduced to this end. Superefficiency does not imply that the error variances of the extracted signals decrease in the order of 1/t/sup 2/ or /spl eta//sup 2/ but implies that their covariances (and independencies) do.

[1]  Shun-ichi Amari,et al.  A Theory of Adaptive Pattern Classifiers , 1967, IEEE Trans. Electron. Comput..

[2]  Dennis J. Clague,et al.  New Classes of Synchronous Codes , 1967, IEEE Trans. Electron. Comput..

[3]  Christian Jutten,et al.  Space or time adaptive signal processing by neural network models , 1987 .

[4]  Jean-Francois Cardoso,et al.  Source separation using higher order moments , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[5]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[6]  Heskes,et al.  Learning processes in neural networks. , 1991, Physical review. A, Atomic, molecular, and optical physics.

[7]  Esfandiar Sorouchyari,et al.  Blind separation of sources, part III: Stability analysis , 1991, Signal Process..

[8]  J. Nadal,et al.  Nonlinear neurons in the low-noise limit: a factorial code maximizes information transfer Network 5 , 1994 .

[9]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[10]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[11]  Erkki Oja,et al.  Signal Separation by Nonlinear Hebbian Learning , 1995 .

[12]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[13]  Opper.  On-line versus Off-line Learning from Random Examples: General Results. , 1996, Physical review letters.

[14]  Reimann,et al.  Unsupervised learning by examples: On-line versus off-line. , 1996, Physical review letters.

[15]  Dinh-Tuan Pham,et al.  Blind separation of instantaneous mixture of sources via an independent component analysis , 1996, IEEE Trans. Signal Process..

[16]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[17]  Shun-ichi Amari,et al.  Blind source separation-semiparametric statistical approach , 1997, IEEE Trans. Signal Process..

[18]  S. Amari,et al.  Estimating Functions in Semiparametric Statistical Models , 1997 .

[19]  Eric Moreau,et al.  Self-adaptive source separation .I. Convergence analysis of a direct linear network controlled by the Herault-Jutten algorithm , 1997, IEEE Trans. Signal Process..

[20]  S.C. Douglas,et al.  Multichannel blind deconvolution and equalization using the natural gradient , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.

[21]  Shun-ichi Amari,et al.  Adaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information , 1997, Neural Computation.

[22]  Shun-ichi Amari,et al.  Stability Analysis Of Adaptive Blind Source Separation , 1997 .

[23]  Andrzej Cichocki,et al.  Stability Analysis of Learning Algorithms for Blind Source Separation , 1997, Neural Networks.

[24]  Philippe Garat,et al.  Blind separation of mixture of independent sources through a quasi-maximum likelihood approach , 1997, IEEE Trans. Signal Process..

[25]  Shun-ichi Amari,et al.  Learning and statistical inference , 1998 .

[26]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[27]  Shun-ichi Amari,et al.  Natural Gradient Learning for Over- and Under-Complete Bases in ICA , 1999, Neural Computation.

[28]  Andrzej Cichocki,et al.  Nonholonomic Orthogonal Learning Algorithms for Blind Source Separation , 2000, Neural Computation.

[29]  O. Macchi,et al.  Reply to "Comments on 'self-adaptive source separation, part I: convergence analysis of a direct linear network controled by the Herault-Jutten algorithm" , 2000, IEEE Trans. Signal Process..