Fundamental Limitations on Search Algorithms: Evolutionary Computing in Perspective

The past twenty years has seen a rapid growth of interest in stochas- tic search algorithms, particularly those inspired by natural processes in physics and biology. Impressive results have been demonstrated on complex practical op- timisation problems and related search applications taken from a variety of fields, but the theoretical understanding of these algorithms remains weak. This results partly from the insufficient attention that has been paid to results showing certain fundamental limitations on universal search algorithms, including the so-called "No Free Lunch" Theorem. This paper extends these results and draws out some of their implications for the design of search algorithms, and for the construction of useful representations. The resulting insights focus attention on tailoring alg- orithms and representations to particular problem classes by exploiting domain knowledge. This highlights the fundamental importance of gaining a better the- oretical grasp of the ways in which such knowledge may be systematically ex- ploited as a major research agenda for the future.

[1]  William E. Hart,et al.  The Role of Development in Genetic Algorithms , 1994, FOGA.

[2]  L. Darrell Whitley,et al.  The Only Challenging Problems Are Deceptive: Global Search by Solving Order-1 Hyperplanes , 1991, ICGA.

[3]  DeceptionSushil J. LouisDepartment Pareto Optimality, Ga-easiness and Deception , 1993 .

[4]  J. David Schaffer,et al.  Proceedings of the third international conference on Genetic algorithms , 1989 .

[5]  L. Darrell Whitley,et al.  Fundamental Principles of Deception in Genetic Search , 1990, FOGA.

[6]  Nicholas J. Radcliffe,et al.  Non-Linear Genetic Representations , 1992, PPSN.

[7]  W. Pinebrook The evolution of strategy. , 1990, Case studies in health administration.

[8]  John J. Grefenstette,et al.  Deception Considered Harmful , 1992, FOGA.

[9]  Fred W. Glover,et al.  Future paths for integer programming and links to artificial intelligence , 1986, Comput. Oper. Res..

[10]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[11]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[12]  Gunar E. Liepins,et al.  Schema Disruption , 1991, ICGA.

[13]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[14]  Sushil J. Louis,et al.  Pareto OptimalityGA-Easiness and Deception (Extended Abstract) , 1993, International Conference on Genetic Algorithms.

[15]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[16]  Terry Jones,et al.  A Model of Landscapes , 1994 .

[17]  D. E. Goldberg,et al.  Genetic Algorithms in Search, Optimization & Machine Learning , 1989 .

[18]  David E. Goldberg,et al.  Genetic Algorithms and Walsh Functions: Part II, Deception and Its Analysis , 1989, Complex Syst..

[19]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[20]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[21]  Nicholas J. Radcliffe,et al.  Equivalence Class Analysis of Genetic Algorithms , 1991, Complex Syst..

[22]  Ingo Rechenberg,et al.  The Evolution Strategy. A Mathematical Model of Darwinian Evolution , 1984 .

[23]  Patrick D. Surry,et al.  Fitness Variance of Formae and Performance Prediction , 1994, FOGA.

[24]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[25]  J. Dupuy,et al.  Synergetics: From Microscopic to Macroscopic Order , 1984 .

[26]  L. Darrell Whitley,et al.  The GENITOR Algorithm and Selection Pressure: Why Rank-Based Allocation of Reproductive Trials is Best , 1989, ICGA.

[27]  Emile H. L. Aarts,et al.  Simulated Annealing: Theory and Applications , 1987, Mathematics and Its Applications.

[28]  J. David Schaffer,et al.  Representation and Hidden Bias: Gray vs. Binary Coding for Genetic Algorithms , 1988, ML.

[29]  Patrick D. Surry,et al.  Formal Memetic Algorithms , 1994, Evolutionary Computing, AISB Workshop.

[30]  Stuart A. Kauffman,et al.  The origins of order , 1993 .

[31]  Yuval Davidor,et al.  Epistasis Variance: Suitability of a Representation to Genetic Algorithms , 1990, Complex Syst..

[32]  John J. Grefenstette,et al.  How Genetic Algorithms Work: A Critical Look at Implicit Parallelism , 1989, ICGA.

[33]  D. Wolpert,et al.  No Free Lunch Theorems for Search , 1995 .

[34]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[35]  John R. Koza,et al.  Evolving a Computer Program to Generate Random Numbers Using the Genetic Programming Paradigm , 1991, ICGA.

[36]  Adam Prügel-Bennett,et al.  A Statistical Mechanical Formulation of the Dynamics of Genetic Algorithms , 1994, Evolutionary Computing, AISB Workshop.

[37]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[38]  Michael D. Vose,et al.  Modeling Simple Genetic Algorithms , 1992, FOGA.

[39]  Andrew J. Mason,et al.  Crossover Non-linearity Ratios and the Genetic Algorithm: Escaping the Blinkers of Schema Processing , 1993 .

[40]  David E. Goldberg,et al.  Genetic Algorithms and Walsh Functions: Part I, A Gentle Introduction , 1989, Complex Syst..