A comprehensive survey of fitness approximation in evolutionary computation

Evolutionary algorithms (EAs) have received increasing interests both in the academy and industry. One main difficulty in applying EAs to real-world applications is that EAs usually need a large number of fitness evaluations before a satisfying result can be obtained. However, fitness evaluations are not always straightforward in many real-world applications. Either an explicit fitness function does not exist, or the evaluation of the fitness is computationally very expensive. In both cases, it is necessary to estimate the fitness function by constructing an approximate model. In this paper, a comprehensive survey of the research on fitness approximation in evolutionary computation is presented. Main issues like approximation levels, approximate model management schemes, model construction techniques are reviewed. To conclude, open questions and interesting issues in the field are discussed.

[1]  Stephane Pierret,et al.  Turbomachinery Blade Design Using a Navier–Stokes Solver and Artificial Neural Network , 1999 .

[2]  Timothy M. Mauery,et al.  COMPARISON OF RESPONSE SURFACE AND KRIGING MODELS FOR MULTIDISCIPLINARY DESIGN OPTIMIZATION , 1998 .

[3]  Kenji Fukumizu,et al.  Statistical active learning in multilayer perceptrons , 2000, IEEE Trans. Neural Networks Learn. Syst..

[4]  Mourad Sefrioui,et al.  A Hierarchical Genetic Algorithm Using Multiple Models for Optimization , 2000, PPSN.

[5]  Wei Shyy,et al.  Response surface and neural network techniques for rocket engine injector optimization , 1999 .

[6]  J. Anderson,et al.  Computational fluid dynamics : the basics with applications , 1995 .

[7]  John J. Grefenstette,et al.  Genetic Search with Approximate Function Evaluation , 1985, ICGA.

[8]  Jürgen Branke,et al.  Creating Robust Solutions by Means of Evolutionary Algorithms , 1998, PPSN.

[9]  Jürgen Branke,et al.  Efficient fitness estimation in noisy environments , 2001 .

[10]  Mark Plutowski,et al.  Selecting concise training sets from clean data , 1993, IEEE Trans. Neural Networks.

[11]  Andy J. Keane,et al.  Optimisation for Multilevel Problems: A Comparison of Various Algorithms , 1998 .

[12]  Manolis Papadrakakis,et al.  Optimization of Large-Scale 3-D Trusses Using Evolution Strategies and Neural Networks , 1999 .

[13]  Gisbert Schneider,et al.  Artificial neural networks and simulated molecular evolution are potential tools for sequence-oriented protein design , 1994, Comput. Appl. Biosci..

[14]  J. -F. M. Barthelemy,et al.  Approximation concepts for optimum structural design — a review , 1993 .

[15]  B. Julstrom,et al.  Design of vector quantization codebooks using a genetic algorithm , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[16]  Min-Jea Tahk,et al.  Acceleration of the convergence speed of evolutionary algorithms using multi-layer neural networks , 2003 .

[17]  David J. C. MacKay,et al.  Information-Based Objective Functions for Active Data Selection , 1992, Neural Computation.

[18]  Sung-Bae Cho,et al.  An efficient genetic algorithm with less fitness evaluation by clustering , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[19]  Bernhard Sendhoff,et al.  On Evolutionary Optimization with Approximate Fitness Functions , 2000, GECCO.

[20]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1995, COLT '90.

[21]  M. H. Choueiki,et al.  Training data development with the D-optimality criterion , 1999, IEEE Trans. Neural Networks.

[22]  Rajkumar Roy,et al.  Multi-objective Optimisation Of Rolling Rod Product Design Using Meta-modelling Approach , 2002, GECCO.

[23]  Dekun Yang,et al.  Evolutionary algorithms with a coarse-to-fine function smoothing , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[24]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[25]  M. Farina.  A neural network based generalized response surface multiobjective evolutionary algorithm , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[26]  G. Schneider,et al.  Peptide design by artificial neural networks and computer-based evolutionary search. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[27]  D. Goldberg,et al.  Don't evaluate, inherit , 2001 .

[28]  A. Keane,et al.  Evolutionary Optimization of Computationally Expensive Problems via Surrogate Modeling , 2003 .

[29]  Khaled Rasheed,et al.  Comparison Of Methods For Using Reduced Models To Speed Up Design Optimization , 2002, GECCO.

[30]  Natalia Alexandrov,et al.  Multidisciplinary design optimization : state of the art , 1997 .

[31]  A. Giotis,et al.  LOW-COST STOCHASTIC OPTIMIZATION FOR ENGINEERING APPLICATIONS , 2002 .

[32]  Giancarlo Mauri,et al.  Application of Evolutionary Algorithms to Protein Folding Prediction , 1997, Artificial Evolution.

[33]  A. J. Booker,et al.  A rigorous framework for optimization of expensive functions by surrogates , 1998 .

[34]  Bernhard Sendhoff,et al.  Structure optimization of neural networks for evolutionary design optimization , 2005, Soft Comput..

[35]  Yaochu Jin,et al.  Adaptive encoding for aerodynamic shape optimization using evolution strategies , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[36]  X. Yao,et al.  Combining landscape approximation and local search in global optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[37]  Somanath Nagendra,et al.  Composite Sandwich Structure Optimization with Application to Satellite Components , 1996 .

[38]  J. Fitzpatrick,et al.  Genetic Algorithms in Noisy Environments , 2005, Machine Learning.

[39]  D. Grierson,et al.  Optimal sizing, geometrical and topological design using a genetic algorithm , 1993 .

[40]  K. S. Anderson,et al.  Genetic crossover strategy using an approximation concept , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[41]  Erik D. Goodman,et al.  Evaluation of Injection Island GA Performance on Flywheel Design Optimisation , 1998 .

[42]  Yaochu Jin,et al.  Knowledge in evolutionary and learning systems , 2002 .

[43]  M. Farina.  A Minimal Cost Hybrid Strategy for Pareto Optimal Front Approximation , 2002 .

[44]  T. W. Layne,et al.  A Comparison of Approximation Modeling Techniques: Polynomial Versus Interpolating Models , 1998 .

[45]  Jongsoo Lee,et al.  Parallel Genetic Algorithm Implementation in Multidisciplinary Rotor Blade Design , 1996 .

[46]  Haym Hirsh,et al.  Informed operators: Speeding up genetic-algorithm-based design optimization using reduced models , 2000, GECCO.

[47]  Andy J. Keane,et al.  Metamodeling Techniques For Evolutionary Optimization of Computationally Expensive Problems: Promises and Limitations , 1999, GECCO.

[48]  Manolis Papadrakakis,et al.  Structural optimization using evolution strategies and neural networks , 1998 .

[49]  Brad Johanson,et al.  GP-Music: An Interactive Genetic Programming System for Music Generation with Automated Fitness Raters , 2007 .

[50]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[51]  David E. Goldberg,et al.  Efficient Discretization Scheduling In Multiple Dimensions , 2002, GECCO.

[52]  Bernhard Sendhoff,et al.  Knowledge Incorporation into Neural Networks From Fuzzy Rules , 2004, Neural Processing Letters.

[53]  Alain Ratle,et al.  Accelerating the Convergence of Evolutionary Algorithms by Fitness Landscape Approximation , 1998, PPSN.

[54]  Gisbert Schneider,et al.  Neural networks are useful tools for drug design , 2000, Neural Networks.

[55]  Peter Bradshaw,et al.  Calculation of compressible turbulent boundary layers on straight-tapered swept wings , 1976 .

[56]  Raphael T. Haftka,et al.  Optimization and Experiments: A Survey , 1998 .

[57]  J. Redmond,et al.  Actuator placement based on reachable set optimization for expected disturbance , 1996 .

[58]  A. Ratle.  Optimal sampling strategies for learning a fitness model , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[59]  Sethu Vijayakumar,et al.  Improving Generalization Ability through Active Learning , 1999 .

[60]  John A. Biles,et al.  GenJam: A Genetic Algorithm for Generating Jazz Solos , 1994, ICMC.

[61]  T. Simpson,et al.  Comparative studies of metamodeling techniques under multiple modeling criteria , 2000 .

[62]  Hajime Kita,et al.  Optimization of Noisy Fitness Functions by Means of Genetic Algorithms Using History of Search , 2000, PPSN.

[63]  Bernhard Sendhoff,et al.  Evolutionary Optimization for Problem Classes with Lamarckian Inheritance , 2000 .

[64]  David E. Goldberg,et al.  Fitness Inheritance In Multi-objective Optimization , 2002, GECCO.

[65]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[66]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[67]  W. Carpenter,et al.  A comparison of polynomial approximations and artificial neural nets as response surfaces , 1993 .

[68]  M. J. D. Powell,et al.  Radial basis functions for multivariable interpolation: a review , 1987 .

[69]  Thomas Bäck,et al.  Metamodel-Assisted Evolution Strategies , 2002, PPSN.

[70]  Yaochu Jin,et al.  Managing approximate models in evolutionary aerodynamic design optimization , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[71]  X. Yao.  Evolutionary Search of Approximated N-dimensional Landscapes , 2000 .

[72]  L. Breiman.  USING ADAPTIVE BAGGING TO DEBIAS REGRESSIONS , 1999 .

[73]  Larry Bull,et al.  On Model-Based Evolutionary Computation , 1999, Soft Comput..

[74]  William C. Carpenter,et al.  Common Misconceptions about Neural Networks as Approximators , 1994 .

[75]  Arnold Neumaier,et al.  Molecular Modeling of Proteins and Mathematical Prediction of Protein Structure , 1997, SIAM Rev..

[76]  Tetsuo Morimoto,et al.  An intelligent approach for optimal control of fruit-storage process using neural networks and genetic algorithms , 1997 .

[77]  Robert E. Smith,et al.  Fitness inheritance in genetic algorithms , 1995, SAC '95.

[78]  Bernhard Sendhoff,et al.  A framework for evolutionary optimization with approximate fitness functions , 2002, IEEE Trans. Evol. Comput..

[79]  Andy J. Keane,et al.  A case for multi-level optimisation in aeronautical design , 1999, The Aeronautical Journal (1968).