An introduction to simulated evolutionary optimization

Natural evolution is a population-based optimization process. Simulating this process on a computer results in stochastic optimization techniques that can often outperform classical methods of optimization when applied to difficult real-world problems. There are currently three main avenues of research in simulated evolution: genetic algorithms, evolution strategies, and evolutionary programming. Each method emphasizes a different facet of natural evolution. Genetic algorithms stress chromosomal operators. Evolution strategies emphasize behavioral changes at the level of the individual. Evolutionary programming stresses behavioral change at the level of the species. The development of each of these procedures over the past 35 years is described. Some recent efforts in these areas are reviewed.

[1]  E. Thorndike.  On the Organization of Intellect. , 1921 .

[2]  S. Wright,et al.  Evolution in Mendelian Populations. , 1931, Genetics.

[3]  G. Simpson.  The meaning of evolution : a study of the history of life and of its significance for man , 1949 .

[4]  George H. Mealy,et al.  A method for synthesizing sequential circuits , 1955 .

[5]  As Fraser,et al.  Simulation of Genetic Systems by Automatic Digital Computers II. Effects of Linkage on Rates of Advance Under Selection , 1957 .

[6]  As Fraser,et al.  Simulation of Genetic Systems by Automatic Digital Computers VII. Effects of Reproductive Ra'l'e, and Intensity of Selection, on Genetic Structure , 1957 .

[7]  Alex Fraser,et al.  Simulation of Genetic Systems by Automatic Digital Computers I. Introduction , 1957 .

[8]  A. Fraser.  Simulation of Genetic Systems by Automatic Digital Computers VI. Epistasis , 1960 .

[9]  S. Tax.  The evolution of life , 1960 .

[10]  G. A. Horridge,et al.  Animal species and evolution. , 1964 .

[11]  H. Bremermann,et al.  AN EVOLUTION-TYPE SEARCH METHOD FOR CONVEX SETS. , 1964 .

[12]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[13]  Hans J. Bremermann,et al.  Quantitative Aspects of Goal-Seeking Self-Organizing Systems* , 1967 .

[14]  J. Reed,et al.  Simulation of biological evolution and machine learning. I. Selection of self-reproducing numeric patterns by data processing machines, effects of hereditary control, mutation type and crossing. , 1967, Journal of theoretical biology.

[15]  Sanborn C. Brown,et al.  Mechanical man : the physical basis of intelligent life , 1968 .

[16]  G. H. Burgin,et al.  On Playing Two-Person Zero-Sum Games against Nonminimax Players , 1969, IEEE Trans. Syst. Sci. Cybern..

[17]  M. Conrad,et al.  Evolution experiments with an artificial ecosystem. , 1970, Journal of theoretical biology.

[18]  H. Bremermann.  On the Dynamics and Trajectories of Evolution Processes , 1973 .

[19]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[20]  George H. Burgin,et al.  Systems Identification by Quasilinearization and by Evolutionary Programming , 1973 .

[21]  P. Werbos,et al.  Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences , 1974 .

[22]  R. Lewontin,et al.  The Genetic Basis of Evolutionary Change , 2022 .

[23]  M. Conrad.  Evolutionary learning circuits. , 1974, Journal of theoretical biology.

[24]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[25]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[26]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[27]  S. Stanley.  A theory of evolution above the species level. , 1975, Proceedings of the National Academy of Sciences of the United States of America.

[28]  J. Crow.  The genetic basis of evolutionary change , 1975 .

[29]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[30]  J. W. Atmar,et al.  Speculation on the evolution of intelligence and its possible realization in machine form. , 1976 .

[31]  H. P. Schwefel,et al.  Numerische Optimierung von Computermodellen mittels der Evo-lutionsstrategie , 1977 .

[32]  M. S. Bazaraa,et al.  Nonlinear Programming , 1979 .

[33]  Roger J.-B. Wets,et al.  Minimization by Random Search Techniques , 1981, Math. Oper. Res..

[34]  E. Mayr.  The Growth of Biological Thought: Diversity, Evolution, and Inheritance , 1983 .

[35]  J. Murray,et al.  The Evolutionary Process , 1985 .

[36]  James L. McClelland,et al.  James L. McClelland, David Rumelhart and the PDP Research Group, Parallel distributed processing: explorations in the microstructure of cognition . Vol. 1. Foundations . Vol. 2. Psychological and biological models . Cambridge MA: M.I.T. Press, 1987. , 1989, Journal of Child Language.

[37]  Brian D. O. Anderson,et al.  Stability of adaptive systems: passivity and averaging analysis , 1986 .

[38]  John J. Grefenstette,et al.  Optimization of Control Parameters for Genetic Algorithms , 1986, IEEE Transactions on Systems, Man, and Cybernetics.

[39]  T. Malthus.  An essay on the principle of population, as it affects the future improvement of society , 2006 .

[40]  Clive Richards,et al.  The Blind Watchmaker , 1987, Bristol Medico-Chirurgical Journal.

[41]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[42]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[43]  Jim Antonisse,et al.  A New Interpretation of Schema Notation that Overtums the Binary Encoding Constraint , 1989, ICGA.

[44]  A. Hoffman.  Arguments on evolution : a paleontologist's perspective , 1989 .

[45]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[46]  John H. Holland,et al.  Concerning the emergence of tag-mediated lookahead in classifier systems , 1990 .

[47]  May C. Chen.  Toward a New Philosophy of Biology: Observations of an Evolutionist , 1990, The Yale Journal of Biology and Medicine.

[48]  Heinz Mühlenbein,et al.  Evolution in Time and Space - The Parallel Genetic Algorithm , 1990, FOGA.

[49]  Riva Wenig Bickel,et al.  Determination of near-optimum use of hospital diagnostic resources using the "GENES" genetic algorithm shell. , 1990, Computers in biology and medicine.

[50]  John R. Koza,et al.  A Hierarchical Approach to Learning the Boolean Multiplexer Function , 1990, FOGA.

[51]  Alden H. Wright,et al.  Genetic Algorithms for Real Parameter Optimization , 1990, FOGA.

[52]  Gunar E. Liepins,et al.  Deceptiveness and Genetic Algorithm Dynamics , 1990, FOGA.

[53]  T. R. Smith,et al.  Genetic design of processing elements for path planning networks , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[54]  L. Darrell Whitley,et al.  Fundamental Principles of Deception in Genetic Search , 1990, FOGA.

[55]  John H. Miller,et al.  Emergent behavior in classifier systems , 1990 .

[56]  Heinz Mühlenbein,et al.  The parallel genetic algorithm as function optimizer , 1991, Parallel Comput..

[57]  José Carlos Príncipe,et al.  A Simulated Annealing Like Convergence Theory for the Simple Genetic Algorithm , 1991, ICGA.

[58]  D. Fogel.  System Identification Through Simulated Evolution: A Machine Learning Approach to Modeling , 1991 .

[59]  Rick L. Riolo,et al.  Modeling Simple Human Category Learning with a Classifier System , 1991, International Conference on Genetic Algorithms.

[60]  D. Fogel.  The evolution of intelligent decision making in gaming , 1991 .

[61]  Frank Hoffmeister,et al.  Scalable Parallelism by Evolutionary Algorithms , 1991 .

[62]  Larry J. Eshelman,et al.  On Crossover as an Evolutionarily Viable Strategy , 1991, ICGA.

[63]  Schloss Birlinghoven.  Evolution in Time and Space -the Parallel Genetic Algorithm , 1991 .

[64]  David B. Fogel,et al.  Meta-evolutionary programming , 1991, [1991] Conference Record of the Twenty-Fifth Asilomar Conference on Signals, Systems & Computers.

[65]  Bernard Manderick,et al.  A Massively Parallel Genetic Algorithm: Implementation and First Analysis , 1991, ICGA.

[66]  G. Syswerda,et al.  Schedule Optimization Using Genetic Algorithms , 1991 .

[67]  William E. Hart,et al.  Optimizing an Arbitrary Function is Hard for the Genetic Algorithm , 1991 .

[68]  Z. Michalewicz,et al.  A genetic algorithm for the linear transportation problem , 1991, IEEE Trans. Syst. Man Cybern..

[69]  Nostrand Reinhold,et al.  the utility of using the genetic algorithm approach on the problem of Davis, L. (1991), Handbook of Genetic Algorithms. Van Nostrand Reinhold, New York. , 1991 .

[70]  Gunar E. Liepins,et al.  Credit assignment and discovery in classifier systems , 1991, Int. J. Intell. Syst..

[71]  D. B. Fogel,et al.  Using evolutionary programming for modeling: an ocean acoustic example , 1992 .

[72]  Kalyanmoy Deb,et al.  Genetic Algorithms, Noise, and the Sizing of Populations , 1992, Complex Syst..

[73]  M. Conrad,et al.  Symbiotic interactions in the EVOLVE III ecosystem model. , 1992, Bio Systems.

[74]  Bruce E. Rosen,et al.  Genetic Algorithms and Very Fast Simulated Reannealing: A comparison , 1992 .

[75]  Gnter Rudolph,et al.  Parallel Approaches to Stochastic Global Optimization , 1992 .

[76]  Kenneth A. De Jong,et al.  Are Genetic Algorithms Function Optimizers? , 1992, PPSN.

[77]  Günter Rudolph,et al.  On Correlated Mutations in Evolution Strategies , 1992, PPSN.

[78]  Andrew Whinston,et al.  Applying Adaptive Credit Assignment Algorithm for the Learning Classifier System Based upon the Genetic Algorithm , 1992 .

[79]  David E. Goldberg,et al.  Control system optimization using genetic algorithms , 1992 .

[80]  John R. Koza.  Hierarchical Automatic Function Definition in Genetic Programming , 1992, FOGA.

[81]  Guy Albert Dumont,et al.  System identification and control using genetic algorithms , 1992, IEEE Trans. Syst. Man Cybern..

[82]  W. Kühn,et al.  Identifikation der Systemparameter 6-achsiger Gelenkarmroboter mit Hilfe der Evolutionsstrategie , 1992, Robotersysteme.

[83]  Michael Conrad.  Molecular computing: the lock-key paradigm , 1992, Computer.

[84]  Stuart H. Rubin,et al.  Case-based learning: A new paradigm for automated knowledge acquisition , 1992 .

[85]  F. G. Pin,et al.  Mobile manipulator configuration optimization using evolutionary programming , 1992 .

[86]  David B. Fogel,et al.  Evolving Behaviors in the Iterated Prisoner's Dilemma , 1993, Evolutionary Computation.

[87]  Hans-Paul Schwefel,et al.  Evolutionary Programming and Evolution Strategies: Similarities and Differences , 1993 .

[88]  Michael de la Maza,et al.  Book review: Genetic Algorithms + Data Structures = Evolution Programs by Zbigniew Michalewicz (Springer-Verlag, 1992) , 1993 .

[89]  Melanie Mitchell,et al.  What makes a problem hard for a genetic algorithm? Some anomalous results and their explanation , 1993, Machine Learning.

[90]  D. Adler,et al.  Genetic algorithms and simulated annealing: a marriage proposal , 1993, IEEE International Conference on Neural Networks.

[91]  O. Fuentes,et al.  Generic algorithms: what fitness scaling is optimal? , 1993 .

[92]  D. B. Fogel,et al.  Using evolutionary programing to create neural networks that are capable of playing tic-tac-toe , 1993, IEEE International Conference on Neural Networks.

[93]  David B. Fogel,et al.  Evolving fuzzy clusters , 1993, IEEE International Conference on Neural Networks.

[94]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[95]  J. R. McDonnell,et al.  Evolving neural network connectivity , 1993, IEEE International Conference on Neural Networks.

[96]  D. Fogel.  ASYMPTOTIC CONVERGENCE PROPERTIES OF GENETIC ALGORITHMS AND EVOLUTIONARY PROGRAMMING: ANALYSIS AND EXPERIMENTS , 1994 .

[97]  Wirt Atmar,et al.  Notes on the simulation of evolution , 1994, IEEE Trans. Neural Networks.

[98]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 2000, Springer Berlin Heidelberg.

[99]  Günter Rudolph,et al.  Convergence analysis of canonical genetic algorithms , 1994, IEEE Trans. Neural Networks.

[100]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[101]  W. Spears,et al.  On the Virtues of Parameterized Uniform Crossover , 1995 .

[102]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[103]  M Conrad,et al.  Evolutionary credit apportionment and its application to time-dependent neural processing. , 1995, Bio Systems.

[104]  David B. Fogel,et al.  Alternative Neural Network Training Methods , 1995, IEEE Expert.

[105]  David E. Goldberg,et al.  Parallel Recombinative Simulated Annealing: A Genetic Algorithm , 1995, Parallel Comput..

[106]  Jean-Yves Potvin,et al.  Genetic Algorithms for the Traveling Salesman Problem , 2005 .