Evaluation-Relaxation Schemes for Genetic and Evolutionary Algorithms

Genetic and evolutionary algorithms have been increasingly applied to solve complex, large scale search problems with mixed success. Competent genetic algorithms have been proposed to solve hard problems quickly, reliably and accurately. They have rendered problems that were difficult to solve by the earlier GAs to be solvable, requiring only a subquadratic number of function evaluations. To facilitate solving large-scale complex problems, and to further enhance the performance of competent GAs, various efficiency-enhancement techniques have been developed. This study investigates one such class of efficiency-enhancement technique called evaluation relaxation. Evaluation-relaxation schemes replace a high-cost, low-error fitness function with a lowcost, high-error fitness function. The error in fitness functions comes in two flavors: Bias and variance. The presence of bias and variance in fitness functions is considered in isolation and strategies for increasing efficiency in both cases are developed. Specifically, approaches for choosing between two fitness functions with either differing variance or differing bias values have been developed. This thesis also investigates fitness inheritance as an evaluationrelaxation scheme. In fitness inheritance, the fitness values of some individuals are inherited from their parents rather than through a costly evaluation function, thereby reducing the total function-evaluation cost. Simple facetwise models have been derived to capture the dynamics in each case and have been verified with simple but illustrating empirical results. These models are also used to develop analytical framework to tune algorithm parameters to obtain maximum speed-up.

[1]  Bernard Manderick,et al.  Fine-Grained Parallel Genetic Algorithms , 1989, ICGA.

[2]  R. Elston The mathematical theory of quantitative genetics , 1982 .

[3]  Jacques Periaux,et al.  Genetic Algorithms in Engineering and Computer Science , 1996 .

[4]  David E. Goldberg,et al.  The Gambler's Ruin Problem, Genetic Algorithms, and the Sizing of Populations , 1999, Evolutionary Computation.

[5]  Andy J. Keane,et al.  Metamodeling Techniques For Evolutionary Optimization of Computationally Expensive Problems: Promises and Limitations , 1999, GECCO.

[6]  Thomas D. LaToza,et al.  On the supply of building blocks , 2001 .

[7]  Natalia Alexandrov,et al.  Multidisciplinary design optimization : state of the art , 1997 .

[8]  David E. Goldberg,et al.  Optimizing Global-Local Search Hybrids , 1999, GECCO.

[9]  David E. Goldberg,et al.  Efficient Genetic Algorithms Using Discretization Scheduling , 2005, Evolutionary Computation.

[10]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[11]  Magnus Rattray,et al.  Noisy Fitness Evaluation in Genetic Algorithms and the Dynamics of Learning , 1996, FOGA.

[12]  Darrell Whitley,et al.  Modeling Hybrid Genetic Algorithms , 1995 .

[13]  George G. Robertson,et al.  Parallel Implementation of Genetic Algorithms in a Classifier Rystem , 1987, ICGA.

[14]  Heinz Mühlenbein,et al.  The Science of Breeding and Its Application to the Breeder Genetic Algorithm (BGA) , 1993, Evolutionary Computation.

[15]  Yaochu Jin,et al.  Managing approximate models in evolutionary aerodynamic design optimization , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[16]  D. Goldberg,et al.  Verification and extension of the theory of global-local hybrids , 2001 .

[17]  Erick Cantú-Paz,et al.  Efficient and Accurate Parallel Genetic Algorithms , 2000, Genetic Algorithms and Evolutionary Computation.

[18]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[19]  Heinz Mühlenbein,et al.  The response to selection equation for skew fitness distributions , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[20]  David E. Goldberg,et al.  Designing Efficient Genetic and Evolutionary Algorithm Hybrids , 2005 .

[21]  Kalyanmoy Deb,et al.  Multimodal Deceptive Functions , 1993, Complex Syst..

[22]  Martina Gorges-Schleuter,et al.  ASPARAGOS An Asynchronous Parallel Genetic Optimization Strategy , 1989, ICGA.

[23]  Thomas Bäck,et al.  Selective Pressure in Evolutionary Algorithms: A Characterization of Selection Mechanisms , 1994, International Conference on Evolutionary Computation.

[24]  David E. Goldberg,et al.  Genetic Algorithms, Tournament Selection, and the Effects of Noise , 1995, Complex Syst..

[25]  B. Julstrom,et al.  Design of vector quantization codebooks using a genetic algorithm , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[26]  Feller William,et al.  An Introduction To Probability Theory And Its Applications , 1950 .

[27]  M. Gorges-Schleuter Asparagos96 and the traveling salesman problem , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[28]  John J. Grefenstette,et al.  Genetic Search with Approximate Function Evaluation , 1985, ICGA.

[29]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[30]  Geoffrey E. Hinton,et al.  How Learning Can Guide Evolution , 1996, Complex Syst..

[31]  Kalyanmoy Deb,et al.  Sufficient conditions for deceptive and easy binary functions , 1994, Annals of Mathematics and Artificial Intelligence.

[32]  K. Mather,et al.  Response to selection. , 1955, Cold Spring Harbor symposia on quantitative biology.

[33]  J. Fitzpatrick,et al.  Genetic Algorithms in Noisy Environments , 2005, Machine Learning.

[34]  J. Fitzpatrick,et al.  Adaptive search space scaling in digital image registration. , 1989, IEEE transactions on medical imaging.

[35]  E. Cant Migration Policies and Takeover Times in Parallel Genetic Algorithms , 1999 .

[36]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .

[37]  van Dh Dick Campen,et al.  Optimization of Multibody Systems Using Approximation Concepts , 1996 .

[38]  David E. Goldberg,et al.  A Survey of Optimization by Building and Using Probabilistic Models , 2002, Comput. Optim. Appl..

[39]  Kalyanmoy Deb,et al.  Analysis of Selection Algorithms: A Markov Chain Approach , 1996, Evolutionary Computation.

[40]  Vladan Babovic,et al.  Genetic Programming, Ensemble Methods and the Bias/Variance Tradeoff - Introductory Investigations , 2000, EuroGP.

[41]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[42]  Dirk Thierens,et al.  Convergence Models of Genetic Algorithm Selection Schemes , 1994, PPSN.

[43]  Bernhard Sendhoff,et al.  On Evolutionary Optimization with Approximate Fitness Functions , 2000, GECCO.

[44]  David E. Goldberg,et al.  Sizing Populations for Serial and Parallel Genetic Algorithms , 1989, ICGA.

[45]  D. Goldberg,et al.  Domino convergence, drift, and the temporal-salience structure of problems , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[46]  Kalyanmoy Deb,et al.  Massive Multimodality, Deception, and Genetic Algorithms , 1992, PPSN.

[47]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[48]  Dirk Thierens,et al.  Mixing in Genetic Algorithms , 1993, ICGA.

[49]  John H. Holland,et al.  Distributed genetic algorithms for function optimization , 1989 .

[50]  David E. Goldberg Using Time Efficiently: Genetic-Evolutionary Algorithms and the Continuation Problem , 1999, GECCO.

[51]  David E. Goldberg,et al.  Genetic Algorithms, Selection Schemes, and the Varying Effects of Noise , 1996, Evolutionary Computation.

[52]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1967 .

[53]  Adam Prügel-Bennett,et al.  Modelling the Dynamics of a Steady State Genetic Algorithm , 1999, FOGA.

[54]  Benjamin W. Wah,et al.  Dynamic Control of Genetic Algorithms in a Noisy Environment , 1993, ICGA.

[55]  Dirk Thierens,et al.  Linkage Information Processing In Distribution Estimation Algorithms , 1999, GECCO.

[56]  Kalyanmoy Deb,et al.  RapidAccurate Optimization of Difficult Problems Using Fast Messy Genetic Algorithms , 1993, ICGA.

[57]  H. Grüneberg,et al.  Introduction to quantitative genetics , 1960 .

[58]  D. E. Goldberg,et al.  Simple Genetic Algorithms and the Minimal, Deceptive Problem , 1987 .

[59]  Erick Cantú-Paz,et al.  Selection Intensity in Genetic Algorithm with Generation Gaps , 2000, GECCO.

[60]  David E. Goldberg,et al.  Genetic Algorithms and Walsh Functions: Part II, Deception and Its Analysis , 1989, Complex Syst..

[61]  Günter Rudolph,et al.  Takeover Times and Probabilities of Non-Generational Selection Rules , 2000, GECCO.

[62]  John H. Holland,et al.  Genetic Algorithms and the Optimal Allocation of Trials , 1973, SIAM J. Comput..

[63]  J. -F. M. Barthelemy,et al.  Approximation concepts for optimum structural design — a review , 1993 .

[64]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[65]  David E. Goldberg,et al.  The Race, the Hurdle, and the Sweet Spot , 1998 .

[66]  L. Darrell Whitley,et al.  Optimization Using Distributed Genetic Algorithms , 1990, PPSN.

[67]  Lothar Thiele,et al.  A Comparison of Selection Schemes Used in Evolutionary Algorithms , 1996, Evolutionary Computation.

[68]  D. Goldberg,et al.  A practical schema theorem for genetic algorithm design and tuning , 2001 .

[69]  David E. Goldberg,et al.  Genetic Algorithms and the Variance of Fitness , 1991, Complex Syst..

[70]  Alain Ratle,et al.  Accelerating the Convergence of Evolutionary Algorithms by Fitness Landscape Approximation , 1998, PPSN.

[71]  Adam Prügel-Bennett,et al.  A Statistical Mechanical Formulation of the Dynamics of Genetic Algorithms , 1994, Evolutionary Computing, AISB Workshop.

[72]  Erick Cantú-Paz Designing Efficient and Accurate Parallel Genetic Algorithms , 1999 .

[73]  Thomas Bäck,et al.  Generalized Convergence Models for Tournament- and (mu, lambda)-Selection , 1995, ICGA.

[74]  David E. Goldberg,et al.  The Design of Innovation: Lessons from and for Competent Genetic Algorithms , 2002 .

[75]  Paul Bryant Grosso,et al.  Computer Simulations of Genetic Adaptation: Parallel Subcomponent Interaction in a Multilocus Model , 1985 .

[76]  Dirk Thierens,et al.  Elitist recombination: an integrated selection recombination GA , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[77]  Lawrence Davis,et al.  Shall We Repair? Genetic AlgorithmsCombinatorial Optimizationand Feasibility Constraints , 1993, ICGA.

[78]  David E. Goldberg,et al.  Takeover Time in a Noisy Environment , 1997, ICGA.

[79]  G. Harik Linkage Learning via Probabilistic Modeling in the ECGA , 1999 .

[80]  Chrisila C. Pettey,et al.  A Theoretical Investigation of a Parallel Genetic Algorithm , 1989, ICGA.

[81]  Benjamin W. Wah,et al.  Scheduling of Genetic Algorithms in a Noisy Environment , 1994, Evolutionary Computation.

[82]  Jim Smith,et al.  Replacement Strategies in Steady State Genetic Algorithms: Static Environments , 1998, FOGA.

[83]  Robert E. Smith,et al.  Fitness inheritance in genetic algorithms , 1995, SAC '95.

[84]  John J. Grefenstette,et al.  A Parallel Genetic Algorithm , 1987, ICGA.

[85]  Kalyanmoy Deb,et al.  Analyzing Deception in Trap Functions , 1992, FOGA.

[86]  Thomas Bck Generalized convergence models for tournament|and (1; ?)|selection , 1995 .

[87]  Hillol Kargupta,et al.  The Gene Expression Messy Genetic Algorithm , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[88]  David E. Goldberg,et al.  Optimal Sampling For Genetic Algorithms , 1996 .

[89]  Erik D. Goodman,et al.  Investigating Parallel Genetic Algorithms on Job Shop Scheduling Problems , 1997, Evolutionary Programming.

[90]  Dirk Thierens,et al.  Toward a Better Understanding of Mixing in Genetic Algorithms , 1993 .

[91]  Milton Abramowitz,et al.  Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables , 1964 .

[92]  Brad L. Miller,et al.  Noise, sampling, and efficient genetic algorthms , 1997 .

[93]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[94]  Kalyanmoy Deb,et al.  Genetic Algorithms, Noise, and the Sizing of Populations , 1992, Complex Syst..

[95]  S. Finch Gambler's Ruin , 2022 .

[96]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[97]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[98]  John J. Grefenstette,et al.  How Genetic Algorithms Work: A Critical Look at Implicit Parallelism , 1989, ICGA.

[99]  Dragan Cvetkovic,et al.  The Optimal Population Size for Uniform Crossover and Truncation Selection , 1994 .

[100]  Erick Cantú-Paz,et al.  A Summary of Research on Parallel Genetic Algorithms , 1995 .

[101]  Lothar Thiele,et al.  A Mathematical Analysis of Tournament Selection , 1995, ICGA.

[102]  David E. Goldberg,et al.  Linkage Identification by Non-monotonicity Detection for Overlapping Functions , 1999, Evolutionary Computation.

[103]  David E. Goldberg,et al.  The compact genetic algorithm , 1999, IEEE Trans. Evol. Comput..

[104]  A. D. Bethke,et al.  Comparison of genetic algorithms and gradient-based optimizers on parallel processors : efficiency of use of processing capacity , 1976 .

[105]  James E. Baker,et al.  Adaptive Selection Methods for Genetic Algorithms , 1985, International Conference on Genetic Algorithms.

[106]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[107]  D. Goldberg,et al.  Don't evaluate, inherit , 2001 .

[108]  Colin R. Reeves,et al.  Using Genetic Algorithms with Small Populations , 1993, ICGA.