Benchmarking evolutionary algorithms for single objective real-valued constrained optimization - A critical review

Benchmarking plays an important role in the development of novel search algorithms as well as for the assessment and comparison of contemporary algorithmic ideas. This paper presents common principles that need to be taken into account when considering benchmarking problems for constrained optimization. Current benchmark environments for testing Evolutionary Algorithms are reviewed in the light of these principles. Along with this line, the reader is provided with an overview of the available problem domains in the field of constrained benchmarking. Hence, the review supports algorithms developers with information about the merits and demerits of the available frameworks.

[1]  Carlos A. Coello Coello,et al.  What Makes a Constrained Problem Difficult to Solve by an Evolutionary Algorithm , 2004 .

[2]  P. Suganthan,et al.  Problem Definitions and Evaluation Criteria for the CEC 2010 Competition on Constrained Real- Parameter Optimization , 2010 .

[3]  Stefan M. Wild,et al.  Benchmarking Derivative-Free Optimization Algorithms , 2009, SIAM J. Optim..

[4]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[5]  V. Klee,et al.  HOW GOOD IS THE SIMPLEX ALGORITHM , 1970 .

[6]  Josef Tvrdík,et al.  A simple framework for constrained problems with application of L-SHADE44 and IDE , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[7]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[8]  Marco Tomassini,et al.  Evolutionary Algorithms , 1995, Towards Evolvable Hardware.

[9]  David S. Johnson,et al.  A theoretician's guide to the experimental analysis of algorithms , 1999, Data Structures, Near Neighbor Searches, and Methodology.

[10]  Zbigniew Michalewicz,et al.  Test-case generator for nonlinear continuous parameter optimization techniques , 2000, IEEE Trans. Evol. Comput..

[11]  Anne Auger,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions , 2009 .

[12]  Andrew M. Sutton,et al.  Differential evolution and non-separability: using selective pressure to focus search , 2007, GECCO '07.

[13]  Tetsuyuki Takahama,et al.  Constrained optimization by the ε constrained differential evolution with an archive and gradient-based mutation , 2010, IEEE Congress on Evolutionary Computation.

[14]  S. T. Buckland,et al.  An Introduction to the Bootstrap , 1994 .

[15]  David Mautner Himmelblau,et al.  Applied Nonlinear Programming , 1972 .

[16]  Susan A. Murphy,et al.  Monographs on statistics and applied probability , 1990 .

[17]  L. Darrell Whitley,et al.  Evaluating Evolutionary Algorithms , 1996, Artif. Intell..

[18]  Nimrod Megiddo,et al.  Boundary Behavior of Interior Point Algorithms in Linear Programming , 1989, Math. Oper. Res..

[20]  Bryan A. Tolson,et al.  A benchmarking framework for simulation-based optimization of environmental models , 2012, Environ. Model. Softw..

[21]  Tamás Terlaky,et al.  How good are interior point methods? Klee–Minty cubes tighten iteration-complexity bounds , 2008, Math. Program..

[22]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[23]  Raymond Ros,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Experimental Setup , 2009 .

[24]  Carlos A. Coello Coello,et al.  Constraint-handling in nature-inspired numerical optimization: Past, present and future , 2011, Swarm Evol. Comput..

[25]  Andreas Ritter,et al.  Handbook Of Test Problems In Local And Global Optimization , 2016 .

[26]  Dan Boneh,et al.  On genetic algorithms , 1995, COLT '95.

[27]  Francisco Herrera,et al.  Since CEC 2005 competition on real-parameter optimisation: a decade of research, progress and comparative analysis’s weakness , 2017, Soft Comput..

[28]  Anne Auger,et al.  COCO: Performance Assessment , 2016, ArXiv.

[29]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[30]  Rajkumar Roy,et al.  Evolutionary computing in manufacturing industry: an overview of recent applications , 2005, Appl. Soft Comput..

[31]  Tetsuyuki Takahama,et al.  Constrained Optimization by the ε Constrained Differential Evolution with Gradient-Based Mutation and Feasible Elites , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[32]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[33]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms for Constrained Parameter Optimization Problems , 1996, Evolutionary Computation.

[34]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[35]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[36]  Zbigniew Michalewicz,et al.  Evolutionary algorithms for constrained engineering problems , 1996, Computers & Industrial Engineering.

[37]  A. Kai Qin,et al.  Self-adaptive Differential Evolution Algorithm for Constrained Real-Parameter Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[38]  Anne Auger,et al.  COCO: a platform for comparing continuous optimizers in a black-box setting , 2016, Optim. Methods Softw..

[39]  Reha Uzsoy,et al.  Experimental Evaluation of Heuristic Optimization Algorithms: A Tutorial , 2001, J. Heuristics.

[40]  Marc Schoenauer,et al.  Multidisciplinary Optimization in the Design of Future Space Launchers , 2013 .

[41]  K. Arrow.  A Difficulty in the Concept of Social Welfare , 1950, Journal of Political Economy.

[42]  Sébastien Le Digabel,et al.  A Taxonomy of Constraints in Simulation-Based Optimization , 2015, 1505.07881.

[43]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms, Homomorphous Mappings, and Constrained Parameter Optimization , 1999, Evolutionary Computation.

[44]  Radka Polakova,et al.  L-SHADE with competing strategies applied to constrained optimization , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[45]  K. Deb.  An Efficient Constraint Handling Method for Genetic Algorithms , 2000 .

[46]  Nicholas I. M. Gould,et al.  CUTEst: a Constrained and Unconstrained Testing Environment with safe threads for mathematical optimization , 2013, Computational Optimization and Applications.

[47]  Hans-Georg Beyer,et al.  A Covariance Matrix Self-Adaptation Evolution Strategy for Linear Constrained Optimization , 2018, ArXiv.

[48]  Anne Auger,et al.  COCO: The Experimental Procedure , 2016, ArXiv.

[49]  Bernd Bischl,et al.  Analyzing the BBOB Results by Means of Benchmarking Concepts , 2015, Evolutionary Computation.

[50]  Wenyin Gong,et al.  Engineering optimization by means of an improved constrained differential evolution , 2014 .

[51]  Jing J. Liang,et al.  Problem Deflnitions and Evaluation Criteria for the CEC 2006 Special Session on Constrained Real-Parameter Optimization , 2006 .

[52]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[53]  Giovanni Squillero,et al.  Applications of Evolutionary Computation , 2016, Lecture Notes in Computer Science.

[54]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[55]  Thomas Stützle,et al.  A Note on Bound Constraints Handling for the IEEE CEC’05 Benchmark Function Suite , 2014, Evolutionary Computation.

[56]  Martin Pelikan,et al.  An introduction and survey of estimation of distribution algorithms , 2011, Swarm Evol. Comput..

[57]  Ponnuthurai N. Suganthan,et al.  Differential evolution with ensemble of constraint handling techniques for solving CEC 2010 benchmark problems , 2010, IEEE Congress on Evolutionary Computation.

[58]  Jun Zhang,et al.  Evolutionary Computation Meets Machine Learning: A Survey , 2011, IEEE Computational Intelligence Magazine.

[59]  Federico Divina,et al.  Applications of Evolutionary Computation - 18th European Conference, EvoApplications , 2015 .