Comprehensive comparison of convergence performance of optimization algorithms based on nonparametric statistical tests

In evolutionary computation, statistical tests are commonly used to improve the comparative evaluation process of the performance of different algorithms. In this paper, three state-of-the-art Differential Evolution (DE) based algorithms, namely Dynamic Memetic Differential Evolution (MOS), Self-adaptive DE hybridized with modified multi-trajectory search (MMTS) algorithm (SaDE-MMTS) and Self-adaptive Differential Evolution Algorithm using Population Size Reduction and three Strategies Algorithm (jDElscop) as well as a novel algorithm called ensemble of parameters and mutation strategies in Differential Evolution with Self-adaption and MMTS (Sa-EPSDE-MMTS), are tested on the most recent LSO benchmark problems and comparatively evaluated using nonparametric statistical analysis. Instead of using the “Value-to-Reach” as the comparison criterion, comprehensive comparison over multiple evolution points are investigated on each test problem in order to quantitatively compare convergence performance of different algorithms. Our investigations demonstrate that even though all these algorithms yield the same final solutions on a large set of problems, they possess statistically significant variations during the convergence. Hence, we propose that evolutionary algorithms can be compared statistically along the evolution paths.

[1]  Ponnuthurai Nagaratnam Suganthan,et al.  Benchmark Functions for the CEC'2013 Special Session and Competition on Large-Scale Global Optimization , 2008 .

[2]  Ponnuthurai N. Suganthan,et al.  Self-adaptive differential evolution with multi-trajectory search for large-scale optimization , 2011, Soft Comput..

[3]  M. Friedman.  A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[4]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[5]  M. Friedman.  The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[6]  Mehmet Fatih Tasgetiren,et al.  Differential evolution algorithm with ensemble of parameters and mutation strategies , 2011, Appl. Soft Comput..

[7]  Janez Brest,et al.  Self-adaptive differential evolution algorithm using population size reduction and three strategies , 2011, Soft Comput..

[8]  Antonio LaTorre,et al.  A MOS-based dynamic memetic differential evolution algorithm for continuous optimization: a scalability test , 2011, Soft Comput..

[9]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[10]  Philip H. Ramsey.  Nonparametric Statistical Methods , 1974, Technometrics.

[11]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[12]  W. Kruskal,et al.  Use of Ranks in One-Criterion Variance Analysis , 1952 .

[13]  Kenneth V. Price,et al.  An introduction to differential evolution , 1999 .

[14]  P. N. Suganthan,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[15]  J. L. Hodges,et al.  Rank Methods for Combination of Independent Experiments in Analysis of Variance , 1962 .

[16]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[17]  Bruce A. Robinson,et al.  Self-Adaptive Multimethod Search for Global Optimization in Real-Parameter Spaces , 2009, IEEE Transactions on Evolutionary Computation.