Adaptive blind signal processing-neural network approaches

Learning algorithms and underlying basic mathematical ideas are presented for the problem of adaptive blind signal processing, especially instantaneous blind separation and multichannel blind deconvolution/equalization of independent source signals. We discuss developments of adaptive learning algorithms based on the natural gradient approach and their properties concerning convergence, stability, and efficiency. Several promising schemas are proposed and reviewed in the paper. Emphasis is given to neural networks or adaptive filtering models and associated online adaptive nonlinear learning algorithms. Computer simulations illustrate the performances of the developed algorithms. Some results presented in this paper are new and are being published for the first time.

[1]  Shun-ichi Amari,et al.  A Theory of Adaptive Pattern Classifiers , 1967, IEEE Trans. Electron. Comput..

[2]  A. Benveniste,et al.  Robust identification of a nonminimum phase system: Blind adjustment of a linear equalizer in data communications , 1980 .

[3]  P. Kumar,et al.  Theory and practice of recursive identification , 1985, IEEE Transactions on Automatic Control.

[4]  Michael G. Larimore,et al.  New processing techniques based on the constant modulus adaptive algorithm , 1985, IEEE Trans. Acoust. Speech Signal Process..

[5]  Lang Tong,et al.  Indeterminacy and identifiability of blind identification , 1991 .

[6]  Christian Jutten,et al.  Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture , 1991, Signal Process..

[7]  C.R. Johnson,et al.  Admissibility in blind adaptive channel equalization , 1991, IEEE Control Systems.

[8]  John C. Platt,et al.  Networks for the Separation of Sources that Are Superimposed and Delayed , 1991, NIPS.

[9]  Dinh Tuan Pham,et al.  Separation of a mixture of independent sources through a maximum likelihood approach , 1992 .

[10]  Andrzej Cichocki,et al.  New learning algorithm for blind separation of sources , 1992 .

[11]  Ehud Weinstein,et al.  New criteria for blind deconvolution of nonminimum phase systems (channels) , 1990, IEEE Trans. Inf. Theory.

[12]  Andrzej Cichocki,et al.  Neural networks for optimization and signal processing , 1993 .

[13]  Andrzej Cichocki,et al.  Robust estimation of principal components by using neural network learning algorithms , 1993 .

[14]  Meir Feder,et al.  Multi-channel signal separation by decorrelation , 1993, IEEE Trans. Speech Audio Process..

[15]  B. Widrow,et al.  Adaptive inverse control , 1987, Proceedings of 8th IEEE International Symposium on Intelligent Control.

[16]  Lang Tong,et al.  Waveform-preserving blind estimation of multiple independent sources , 1993, IEEE Trans. Signal Process..

[17]  Juha Karhunen,et al.  Learning of robust principal component subspace , 1993, Proceedings of 1993 International Conference on Neural Networks (IJCNN-93-Nagoya, Japan).

[18]  Ehud Weinstein,et al.  Super-exponential methods for blind deconvolution , 1993, IEEE Trans. Inf. Theory.

[19]  D. Slock.  Blind joint equalization of multiple synchronous mobile users using oversampling and/or multiple antennas , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[20]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[21]  Lang Tong,et al.  Blind identification and equalization based on second-order statistics: a time domain approach , 1994, IEEE Trans. Inf. Theory.

[22]  Andrzej Cichocki,et al.  Robust learning algorithm for blind separation of signals , 1994 .

[23]  T. Kailath,et al.  A least-squares approach to blind channel identification , 1995, IEEE Trans. Signal Process..

[24]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[25]  Christian Jutten,et al.  Blind source separation for convolutive mixtures , 1995, Signal Process..

[26]  Erkki Oja,et al.  Principal and Independent Components in Neural Networks - Recent Developments , 1995 .

[27]  Jean-Louis Lacoume,et al.  Blind separation of wide-band sources in the frequency domain , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[28]  Erkki Oja,et al.  Signal Separation by Nonlinear Hebbian Learning , 1995 .

[29]  Juha Karhunen,et al.  Nonlinear PCA type approaches for source separation and independent component analysis , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[30]  Nathalie Delfosse,et al.  Adaptive blind separation of independent sources: A deflation approach , 1995, Signal Process..

[31]  Kiyotoshi Matsuoka,et al.  A neural net for blind separation of nonstationary signals , 1995, Neural Networks.

[32]  Shun-ichi Amari,et al.  Recurrent Neural Networks For Blind Separation of Sources , 1995 .

[33]  Chrysostomos L. Nikias,et al.  EVAM: an eigenvector-based algorithm for multichannel blind deconvolution of input colored signals , 1995, IEEE Trans. Signal Process..

[34]  Haim Sompolinsky,et al.  On-line Learning of Dichotomies: Algorithms and Learning Curves. , 1995, NIPS 1995.

[35]  Andrzej Cichocki,et al.  A New Learning Algorithm for Blind Signal Separation , 1995, NIPS.

[36]  Kari Torkkola,et al.  Blind separation of convolved sources based on information maximization , 1996, Neural Networks for Signal Processing VI. Proceedings of the 1996 IEEE Signal Processing Society Workshop.

[37]  Philippe Loubaton,et al.  Second order blind equalization in multiple input multiple output FIR systems: a weighted least squares approach , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[38]  Jitendra K. Tugnait,et al.  Blind equalization and channel estimation for multiple-input multiple-output communications systems , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[39]  J. Príncipe,et al.  Temporal decorrelation using teacher forcing anti-Hebbian learning and its application in adaptive blind source separation , 1996, Neural Networks for Signal Processing VI. Proceedings of the 1996 IEEE Signal Processing Society Workshop.

[40]  John J. Shynk,et al.  The constant modulus array for cochannel signal copy and direction finding , 1996, IEEE Trans. Signal Process..

[41]  Eric Moreau,et al.  High order contrasts for self-adaptive source separation criteria for complex source separation , 1996 .

[42]  Shun-ichi Amari,et al.  Adaptive approach to blind source separation with cancellation of additive and convolutional noise , 1996, Proceedings of Third International Conference on Signal Processing (ICSP'96).

[43]  Yingbo Hua,et al.  Fast maximum likelihood for blind identification of multiple FIR channels , 1996, IEEE Trans. Signal Process..

[44]  Shun-ichi Amari,et al.  Self-adaptive neural networks for blind separation of sources , 1996, 1996 IEEE International Symposium on Circuits and Systems. Circuits and Systems Connecting the World. ISCAS 96.

[45]  Stefaan Van Gerven,et al.  Adaptive noise cancellation and signal separation with applications to speech enhancement , 1996 .

[46]  Barak A. Pearlmutter,et al.  A Context-Sensitive Generalization of ICA , 1996 .

[47]  P. Comon,et al.  Contrasts for multichannel blind deconvolution , 1996, IEEE Signal Processing Letters.

[48]  S. Amari,et al.  Fast-convergence filtered regressor algorithms for blind equalisation , 1996 .

[49]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[50]  Andrzej Cichocki,et al.  Neural network approach to blind separation and enhancement of images , 1996, 1996 8th European Signal Processing Conference (EUSIPCO 1996).

[51]  Juha Karhunen,et al.  Neural approaches to independent component analysis and source separation , 1996, ESANN.

[52]  Ruey-Wen Liu,et al.  General approach to blind source separation , 1996, IEEE Trans. Signal Process..

[53]  Simon J. Godsill,et al.  Multi-channel signal separation , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[54]  Andrzej Cichocki,et al.  Robust neural networks with on-line learning for blind identification and blind separation of sources , 1996 .

[55]  Andreas Ziehe,et al.  Adaptive On-line Learning in Changing Environments , 1996, NIPS.

[56]  Erkki Oja,et al.  One-unit Learning Rules for Independent Component Analysis , 1996, NIPS.

[57]  Te-Won Lee,et al.  Blind Separation of Delayed and Convolved Sources , 1996, NIPS.

[58]  Ehud Weinstein,et al.  Multichannel signal separation: methods and analysis , 1996, IEEE Trans. Signal Process..

[59]  Shun-ichi Amari,et al.  Blind source separation-semiparametric statistical approach , 1997, IEEE Trans. Signal Process..

[60]  Erkki Oja,et al.  The nonlinear PCA learning rule in independent component analysis , 1997, Neurocomputing.

[61]  Colin Fyfe,et al.  Independence is far from normal , 1997, ESANN.

[62]  Yannick Deville.  Analysis of the convergence properties of a self-normalized source separation neural network , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.

[63]  Colin Fyfe,et al.  Kurtosis extrema and identification of independent components: a neural network approach , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[64]  Ruey-Wen Liu,et al.  A fundamental theorem for multiple-channel blind equalization , 1997 .

[65]  C. Papadias,et al.  A constant modulus algorithm for multiuser signal separation in presence of delay spread using antenna arrays , 1997, IEEE Signal Processing Letters.

[66]  Andrzej Cichocki,et al.  Modified Herault-Jutten Algorithms for Blind Separation of Sources , 1997, Digit. Signal Process..

[67]  Eric Moreau,et al.  Self-adaptive source separation .I. Convergence analysis of a direct linear network controlled by the Herault-Jutten algorithm , 1997, IEEE Trans. Signal Process..

[68]  T. Sato,et al.  On-line algorithms for blind deconvolution of multichannel linear time-invariant systems , 1997, Proceedings of the IEEE Signal Processing Workshop on Higher-Order Statistics.

[69]  Shun-ichi Amari,et al.  Sequential blind signal extraction in order specified by stochastic properties , 1997 .

[70]  Y. Deville,et al.  Analysis of the convergence properties of a self-normalized source separation neural network , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.

[71]  S.C. Douglas,et al.  Multichannel blind deconvolution and equalization using the natural gradient , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.

[72]  Juha Karhunen,et al.  On Neural Blind Separation with Noise Suppression and Redundancy Reduction , 1997, Int. J. Neural Syst..

[73]  Shun-ichi Amari,et al.  Adaptive Online Learning Algorithms for Blind Separation: Maximum Entropy and Minimum Mutual Information , 1997, Neural Computation.

[74]  S. Amari,et al.  Nonlinearity and Separation Capability: Further Justiication for the Ica Algorithm with a Learned Mixture of Parametric Densities , 1997 .

[75]  Colin Fyfe,et al.  Stochastic ICA Contrast Maximisation Using Oja's Nonlinear PCA Algorithm , 1997, Int. J. Neural Syst..

[76]  Shun-ichi Amari,et al.  On-line adaptive algorithms in non-stationary environments using a modified conjugate gradient approach , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[77]  Shun-ichi Amari,et al.  Stability Analysis Of Adaptive Blind Source Separation , 1997 .

[78]  J. Cardoso.  Infomax and maximum likelihood for blind source separation , 1997, IEEE Signal Processing Letters.

[79]  Yujiro Inouye,et al.  Cumulant-based blind identification of linear multi-input-multi-output systems driven by colored inputs , 1997, IEEE Trans. Signal Process..

[80]  Andrzej Cichocki,et al.  Stability Analysis of Learning Algorithms for Blind Source Separation , 1997, Neural Networks.

[81]  Andrzej Cichocki,et al.  Neural networks for blind decorrelation of signals , 1997, IEEE Trans. Signal Process..

[82]  S. Amari,et al.  Quasi-Newton filtered-error and filtered-regressor algorithms for adaptive equalization and deconvolution , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.

[83]  Arogyaswami Paulraj,et al.  A subspace approach to blind space-time signal processing for wireless communication systems , 1997, IEEE Trans. Signal Process..

[84]  Shun-ichi Amari,et al.  Neural Network Models for Blind Separation of Time Delayed and Convolved Signals , 1997 .

[85]  S. Amari,et al.  Information geometry of estimating functions in semi-parametric statistical models , 1997 .

[86]  Mark Girolami,et al.  Extraction of independent signal sources using a deflationary exploratory projection pursuit network , 1997 .

[87]  S. Amari,et al.  Multichannel blind separation and deconvolution of sources with arbitrary distributions , 1997, Neural Networks for Signal Processing VII. Proceedings of the 1997 IEEE Signal Processing Society Workshop.

[88]  Philippe Garat,et al.  Blind separation of mixture of independent sources through a quasi-maximum likelihood approach , 1997, IEEE Trans. Signal Process..

[89]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[90]  Seungjin Choi,et al.  Cascade neural networks for multichannel blind deconvolution , 1998 .

[91]  Andrzej Cichocki,et al.  Information-theoretic approach to blind separation of sources in non-linear mixture , 1998, Signal Process..