Algorithm comparisons and the significance of population size

In studies that compare the performance of population-based optimization algorithms, it is sometimes assumed that the comparison is valid as long as the number of function evaluations is equal, even if the population size differs. This paper shows that such comparisons are invalid. The performance of two algorithms: differential evolution (DE) and global best particle swarm optimization (gbest PSO) are tested on standard benchmark problems with different numbers of individuals/particles (20, 50 and 100). It is shown that there are significance differences in the performance of the same algorithm with the same number of function evaluations, but with different numbers of individuals/particles. Comparisons of different algorithms should therefore always use the same population size for results to be valid.

[1]  R. Sarker,et al.  Population size, search space and quality of solution: an experimental study , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[2]  Tughrul Arslan,et al.  Elitist selection schemes for genetic algorithm based printed circuit board inspection system , 2005, 2005 IEEE Congress on Evolutionary Computation.

[3]  Xin Yao,et al.  Stochastic ranking for constrained evolutionary optimization , 2000, IEEE Trans. Evol. Comput..

[4]  David E. Goldberg,et al.  The Gambler's Ruin Problem, Genetic Algorithms, and the Sizing of Populations , 1999, Evolutionary Computation.

[5]  Yanchun Liang,et al.  An improved genetic algorithm with variable population-size and a PSO-GA based hybrid evolutionary algorithm , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[6]  Yang Li,et al.  Nonlinear mapping using real-valued genetic algorithm , 1999 .

[7]  Michael Defoin-Platel,et al.  A versatile quantum-inspired evolutionary algorithm , 2007, 2007 IEEE Congress on Evolutionary Computation.

[8]  William B. Langdon,et al.  Understanding particle swarm optimisation by evolving problem landscapes , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[9]  Gary G. Yen,et al.  A Self Adaptive Penalty Function Based Algorithm for Constrained Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[10]  Gary G. Yen,et al.  Dynamic Population Size in PSO-based Multiobjective Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[11]  Gary G. Yen,et al.  A generic framework for constrained optimization using genetic algorithms , 2005, IEEE Transactions on Evolutionary Computation.

[12]  Zbigniew Michalewicz,et al.  Evolutionary Algorithms, Homomorphous Mappings, and Constrained Parameter Optimization , 1999, Evolutionary Computation.

[13]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[14]  Holger R. Maier,et al.  Minimum Number of Generations Required for Convergence of Genetic Algorithms , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[15]  K. Deb An Efficient Constraint Handling Method for Genetic Algorithms , 2000 .

[16]  Andries Petrus Engelbrecht,et al.  Comparing Particle Swarm Optimisation and Genetic Algorithms for Nonlinear Mapping , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[17]  Rainer Storn,et al.  Minimizing the real functions of the ICEC'96 contest by differential evolution , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[18]  Sana Ben Hamida,et al.  An Adaptive Algorithm for Constrained Optimization Problems , 2000, PPSN.

[19]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[20]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[21]  Jonathan A. Wright,et al.  Self-adaptive fitness formulation for constrained optimization , 2003, IEEE Trans. Evol. Comput..