Hybrid methods using evolutionary algorithms for on-line training

A novel hybrid evolutionary approach is presented in this paper for improving the performance of neural network classifiers in slowly varying environments. For this purpose, we investigate a coupling of differential evolution strategy and stochastic gradient descent, using both the global search capabilities of evolutionary strategies and the effectiveness of online gradient descent. The use of differential evolution strategy is related to the concept of evolution of a number of individuals from generation to generation and that of online gradient descent to the concept of adaptation to the environment by learning. The hybrid algorithm is tested in two real-life image processing applications. Experimental results suggest that the hybrid strategy is capable to train online effectively leading to networks with increased generalization capability.

[1]  Sung Wook Baik,et al.  Adaptive object recognition based on the radial basis function paradigm , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[2]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[3]  R.M. Haralick,et al.  Statistical and structural approaches to texture , 1979, Proceedings of the IEEE.

[4]  PoliRiccardo,et al.  Evolving the Topology and the Weights of Neural Networks Using a Dual Representation , 1998 .

[5]  George D. Magoulas,et al.  Effective Backpropagation Training with Variable Stepsize , 1997, Neural Networks.

[6]  Richard S. Sutton,et al.  Online Learning with Random Representations , 1993, ICML.

[7]  Phil Brodatz,et al.  Textures: A Photographic Album for Artists and Designers , 1966 .

[8]  Graeme G. Wilkinson,et al.  Open Questions in Neurocomputing for Earth Observation , 1997 .

[9]  Thibault Langlois,et al.  Parameter adaptation in stochastic optimization , 1999 .

[10]  Richard S. Sutton,et al.  Adapting Bias by Gradient Descent: An Incremental Version of Delta-Bar-Delta , 1992, AAAI.

[11]  G. D. Magoulas,et al.  Image recognition and neuronal networks: Intelligent systems for the improvement of imaging information , 2000, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[12]  Terence C. Fogarty,et al.  A Comparative Study of Steady State and Generational Genetic Algorithms , 1996, Evolutionary Computing, AISB Workshop.

[13]  Stefanos Kollias,et al.  Neural network-assisted effective lossy compression of medical images , 1996, Proc. IEEE.

[14]  Nicol N. Schraudolph,et al.  Local Gain Adaptation in Stochastic Gradient Descent , 1999 .

[15]  Nicol N. Schraudolph,et al.  Online Local Gain Adaptation for Multi-Layer Perceptrons , 1998 .

[16]  Vassilis P. Plagianakos,et al.  Training neural networks with threshold activation functions and constrained integer weights , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[17]  Peter J. Angeline,et al.  Tracking Extrema in Dynamic Environments , 1997, Evolutionary Programming.

[18]  Ralf Salomon,et al.  Adaptation on the Evolutionary Time Scale: A Working Hypothesis and Basic Experiments , 1997, Artificial Evolution.

[19]  Shumeet Baluja,et al.  Evolution of an artificial neural network based autonomous land vehicle controller , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[20]  Sung Wook Baik,et al.  Adaptive RBF classifier for object recognition in image sequences , 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.

[21]  Vassilis P. Plagianakos,et al.  Learning in multilayer perceptrons using global optimization strategies , 2001 .

[22]  David Saad,et al.  On-Line Learning in Neural Networks , 1999 .

[23]  M. N. Vrahatis,et al.  Adaptive stepsize algorithms for on-line training of neural networks , 2001 .

[24]  Stefanos D. Kollias,et al.  On-line retrainable neural networks: improving the performance of neural networks in image analysis problems , 2000, IEEE Trans. Neural Networks Learn. Syst..

[25]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..