A parameter-free particle swarm optimization algorithm using performance classifiers

Abstract This paper presents an investigation into the short-term versus long-term performance of various particle swarm optimization (PSO) control parameter configurations. While evidence suggests that the best PSO parameter values to employ are time-dependent, this paper provides an in-depth examination of a small set of parameter values to provide a more concrete quantification of the performance degradation observed with specific control parameter configurations over time. Given that the short-term performance is not necessarily indicative of long-term performance, this paper proposes that machine learning techniques be used to build predictive models based on two easily-observable landscape characteristics. Finally, using the predictive models as a basis, this paper also proposes a parameter-free PSO algorithm, which performs on par with other top-performing PSO variants, namely the three best performing static PSO configurations, particle swarm optimization with time-varying acceleration coefficients (PSO-TVAC), and particle swarm optimization with improved random constants (PSO-iRC).

[1]  Andries Petrus Engelbrecht,et al.  Particle swarm convergence: An empirical investigation , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[2]  A. P. Engelbrecht.  Particle Swarm Optimization: Iteration Strategies Revisited , 2013, 2013 BRICS Congress on Computational Intelligence and 11th Brazilian Congress on Computational Intelligence.

[3]  Visakan Kadirkamanathan,et al.  Stability analysis of the particle dynamics in particle swarm optimizer , 2006, IEEE Transactions on Evolutionary Computation.

[4]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[5]  A. P. Engelbrecht,et al.  Particle Swarm Optimization: Global Best or Local Best? , 2013, 2013 BRICS Congress on Computational Intelligence and 11th Brazilian Congress on Computational Intelligence.

[6]  Andries Petrus Engelbrecht,et al.  Particle swarm optimization: Velocity initialization , 2012, 2012 IEEE Congress on Evolutionary Computation.

[7]  J. Shaffer.  Modified Sequentially Rejective Multiple Test Procedures , 1986 .

[8]  Andries Petrus Engelbrecht,et al.  A survey of techniques for characterising fitness landscapes and some possible ways forward , 2013, Inf. Sci..

[9]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[10]  M. Jiang,et al.  Particle Swarm Optimization - Stochastic Trajectory Analysis and Parameter Selection , 2007 .

[11]  D. Broomhead,et al.  Exact analysis of the sampling distribution for the canonical particle swarm optimiser and its convergence during stagnation , 2007, GECCO '07.

[12]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[13]  Andries Petrus Engelbrecht,et al.  Optimal parameter regions for particle swarm optimization algorithms , 2017, 2017 IEEE Congress on Evolutionary Computation (CEC).

[14]  Zbigniew Michalewicz,et al.  Analysis of Stability, Local Convergence, and Transformation Sensitivity of a Variant of the Particle Swarm Optimization Algorithm , 2016, IEEE Transactions on Evolutionary Computation.

[15]  Andries Petrus Engelbrecht,et al.  Analysis and classification of optimisation benchmark functions and benchmark suites , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[16]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[17]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[18]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[19]  Andries Petrus Engelbrecht,et al.  An adaptive particle swarm optimization algorithm based on optimal parameter regions , 2017, 2017 IEEE Symposium Series on Computational Intelligence (SSCI).

[20]  Andries Petrus Engelbrecht,et al.  Inertia weight control strategies for particle swarm optimization , 2016, Swarm Intelligence.

[21]  Qunfeng Liu,et al.  Order-2 Stability Analysis of Particle Swarm Optimization , 2015, Evolutionary Computation.

[22]  Andries Petrus Engelbrecht,et al.  A study of particle swarm optimization particle trajectories , 2006, Inf. Sci..

[23]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[24]  Riccardo Poli,et al.  Mean and Variance of the Sampling Distribution of Particle Swarm Optimizers During Stagnation , 2009, IEEE Transactions on Evolutionary Computation.

[25]  Zbigniew Michalewicz,et al.  Impacts of Coefficients on Movement Patterns in the Particle Swarm Optimization Algorithm , 2017, IEEE Transactions on Evolutionary Computation.

[26]  Andries Petrus Engelbrecht,et al.  The sad state of self-adaptive particle swarm optimizers , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[27]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[28]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[29]  Andries Petrus Engelbrecht,et al.  A generalized theoretical deterministic particle swarm model , 2014, Swarm Intelligence.

[30]  Wei Zhang,et al.  A parameter selection strategy for particle swarm optimization based on particle positions , 2014, Expert Syst. Appl..

[31]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[32]  R. Salomon.  Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms. , 1996, Bio Systems.

[33]  Andries Petrus Engelbrecht,et al.  Optimal parameter regions and the time-dependence of control parameter values for the particle swarm optimization algorithm , 2018, Swarm Evol. Comput..

[34]  Andries Petrus Engelbrecht,et al.  A self-adaptive heterogeneous pso for real-parameter optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[35]  Andries Petrus Engelbrecht,et al.  On the optimality of particle swarm parameters in dynamic environments , 2013, 2013 IEEE Congress on Evolutionary Computation.

[36]  Andries Petrus Engelbrecht,et al.  Measuring exploration/exploitation in particle swarms using swarm diversity , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[37]  Andries Petrus Engelbrecht,et al.  Particle swarm stability: a theoretical extension using the non-stagnate distribution assumption , 2018, Swarm Intelligence.

[38]  Andries Petrus Engelbrecht,et al.  Comparison of self-adaptive particle swarm optimizers , 2014, 2014 IEEE Symposium on Swarm Intelligence.

[39]  E. T. Oldewage,et al.  Degrees of stochasticity in particle swarm optimization , 2019, Swarm Intelligence.

[40]  Andries Petrus Engelbrecht,et al.  Self-adaptive particle swarm optimization: a review and analysis of convergence , 2017, Swarm Intelligence.

[41]  Andries Petrus Engelbrecht,et al.  Particle swarm optimizer: The impact of unstable particles on performance , 2016, 2016 IEEE Symposium Series on Computational Intelligence (SSCI).

[42]  Changhe Li,et al.  A Self-Learning Particle Swarm Optimizer for Global Optimization Problems , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[43]  Andries Petrus Engelbrecht,et al.  Particle swarm variants: standardized convergence analysis , 2015, Swarm Intelligence.

[44]  M. Friedman.  A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[45]  Andries Petrus Engelbrecht,et al.  Ruggedness, funnels and gradients in fitness landscapes and the effect on PSO performance , 2013, 2013 IEEE Congress on Evolutionary Computation.