Fitness function evaluations: A fair stopping condition?

It has become acceptable practice to use only a limit on the number of fitness function evaluations (FEs) as a stopping condition when comparing population-based optimization algorithms, irrespective of the initial number of candidate solutions. This practice has been advocated in a number of competitions to compare the performance of population-based algorithms, and has been used in many articles that contain empirical comparisons of algorithms. This paper advocates the opinion that this practice does not result in fair comparisons, and provides an abundance of empirical evidence to support this claim. Empirical results are obtained from application of a standard global best particle swarm optimization (PSO) algorithm with different swarm sizes under the same FE computational limit, on a large benchmark suite.

[1]  Michael Defoin-Platel,et al.  A versatile quantum-inspired evolutionary algorithm , 2007, 2007 IEEE Congress on Evolutionary Computation.

[2]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[3]  Jonathan A. Wright,et al.  Self-adaptive fitness formulation for constrained optimization , 2003, IEEE Trans. Evol. Comput..

[4]  N. Hansen,et al.  Markov Chain Analysis of Cumulative Step-Size Adaptation on a Linear Constrained Problem , 2015, Evolutionary Computation.

[5]  Ponnuthurai Nagaratnam Suganthan,et al.  Benchmark Functions for the CEC'2013 Special Session and Competition on Large-Scale Global Optimization , 2008 .

[6]  Andries Petrus Engelbrecht,et al.  Comparing Particle Swarm Optimisation and Genetic Algorithms for Nonlinear Mapping , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[7]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[8]  Xin Yao,et al.  Stochastic ranking for constrained evolutionary optimization , 2000, IEEE Trans. Evol. Comput..

[9]  Andries Petrus Engelbrecht,et al.  Particle swarm optimization: Velocity initialization , 2012, 2012 IEEE Congress on Evolutionary Computation.

[10]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[11]  Gary G. Yen,et al.  A Self Adaptive Penalty Function Based Algorithm for Constrained Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[12]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[13]  Andries Petrus Engelbrecht,et al.  Algorithm comparisons and the significance of population size , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[14]  Rolf Wanka,et al.  Theoretical Analysis of Initial Particle Swarm Behavior , 2008, PPSN.

[15]  R. Salomon.  Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms. , 1996, Bio Systems.

[16]  Xiaodong Li,et al.  Benchmark Functions for CEC'2013 Special Session and Competition on Niching Methods for Multimodal Function Optimization' , 2013 .

[17]  Gary G. Yen,et al.  A generic framework for constrained optimization using genetic algorithms , 2005, IEEE Transactions on Evolutionary Computation.

[18]  A. Engelbrecht,et al.  A new locally convergent particle swarm optimiser , 2002, IEEE International Conference on Systems, Man and Cybernetics.