On-line adaptive algorithms in non-stationary environments using a modified conjugate gradient approach

In this paper we propose novel computationally efficient schemas for a large class of online adaptive algorithms with variable self-adaptive learning rates. The learning rate is adjusted automatically providing relatively fast convergence at early stages of adaptation while ensuring small final misadjustment for cases of stationary environments. For nonstationary environments, the algorithms proposed have good tracking ability and quick adaptation to new conditions. Their validity and efficiency are illustrated for a nonstationary blind separation problem.

[1]  Shun-ichi Amari,et al.  A Theory of Adaptive Pattern Classifiers , 1967, IEEE Trans. Electron. Comput..

[2]  M. Srinath,et al.  Conjugate gradient techniques for adaptive filtering , 1992 .

[3]  V. John Mathews,et al.  A stochastic gradient adaptive filter with gradient adaptive step size , 1993, IEEE Trans. Signal Process..

[4]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[5]  J. Príncipe,et al.  Temporal decorrelation using teacher forcing anti-Hebbian learning and its application in adaptive blind source separation , 1996, Neural Networks for Signal Processing VI. Proceedings of the 1996 IEEE Signal Processing Society Workshop.

[6]  Shun-ichi Amari,et al.  Self-adaptive neural networks for blind separation of sources , 1996, 1996 IEEE International Symposium on Circuits and Systems. Circuits and Systems Connecting the World. ISCAS 96.

[7]  Jean-François Cardoso,et al.  Equivariant adaptive source separation , 1996, IEEE Trans. Signal Process..

[8]  Andreas Ziehe,et al.  Adaptive On-line Learning in Changing Environments , 1996, NIPS.

[9]  S.C. Douglas,et al.  Multichannel blind deconvolution and equalization using the natural gradient , 1997, First IEEE Signal Processing Workshop on Signal Processing Advances in Wireless Communications.

[10]  F. Ueng,et al.  Adaptive VSS blind equalizers , 1997, IEEE Signal Processing Letters.

[11]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.