Evolutionary algorithms and gradient search: similarities and differences

Classical gradient methods and evolutionary algorithms represent two very different classes of optimization techniques that seem to have very different properties. This paper discusses some aspects of some "obvious" differences and explores to what extent a hybrid method, the evolutionary-gradient-search procedure, can be used beneficially in the field of continuous parameter optimization. Simulation experiments show that on some test functions, the hybrid method yields faster convergence than pure evolution strategies, but that on other test functions, the procedure exhibits the same deficiencies as steepest-descent methods.

[1]  H. H. Rosenbrock,et al.  An Automatic Method for Finding the Greatest or Least Value of a Function , 1960, Comput. J..

[2]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[3]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[4]  James L. McClelland,et al.  Parallel Distributed Processing: Explorations in the Microstructure of Cognition : Psychological and Biological Models , 1986 .

[5]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[6]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[7]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[8]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[9]  Charles E. Taylor Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. Complex Adaptive Systems.John H. Holland , 1994 .

[10]  Dirk Thierens,et al.  Convergence Models of Genetic Algorithm Selection Schemes , 1994, PPSN.

[11]  H. Muhlenbein,et al.  Gene pool recombination and utilization of covariances for the Breeder Genetic Algorithm , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[12]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[13]  Thomas Bäck,et al.  EVOLUTIONARY ALGORITHMS FOR FUZZY LOGIC: A BRIEF OVERVIEW , 1995 .

[14]  D. Fogel,et al.  A comparison of methods for self-adaptation in evolutionary algorithms. , 1995, Bio Systems.

[15]  R. Salomon Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms. , 1996, Bio Systems.

[16]  J. Leo van Hemmen,et al.  Accelerating backpropagation through dynamic self-adaptation , 1996, Neural Networks.

[17]  N. Hansen,et al.  Convergence Properties of Evolution Strategies with the Derandomized Covariance Matrix Adaptation: T , 1997 .

[18]  H. Beyer An alternative explanation for the manner in which genetic algorithms operate. , 1997, Bio Systems.

[19]  Ralf Salomon Accelerating the Evolutionary-Gradient-Search Procedure: Individual Step Sizes , 1998, PPSN.

[20]  Hans-Georg Beyer On the "Explorative Power" of ES/EP-like Algorithms , 1998, Evolutionary Programming.

[21]  Hans-Paul Schwefel Evolutionary Computation - A Study on Collective Learning , 1998 .