Particle swarm optimisation failure prediction based on fitness landscape characteristics

Particle swarm optimisation (PSO) algorithms have been successfully used to solve many complex real-world optimisation problems. Since their introduction in 1995, the focus of research in PSOs has largely been on the algorithmic side with many new variations proposed on the original PSO algorithm. Relatively little attention has been paid to the study of problems with respect to PSO performance. The aim of this study is to investigate whether a link can be found between problem characteristics and algorithm performance for PSOs. A range of benchmark problems are numerically characterised using fitness landscape analysis techniques. Decision tree induction is used to develop failure prediction models for seven different variations on the PSO algorithm. Results show that for most PSO models, failure could be predicted to a fairly high level of accuracy. The resulting prediction models are not only useful as predictors of failure, but also provide insight into the algorithms themselves, especially when expressed as fuzzy rules in terms of fitness landscape features.

[1]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[2]  Bernd Bischl,et al.  Exploratory landscape analysis , 2011, GECCO '11.

[3]  Andries Petrus Engelbrecht,et al.  Quantifying ruggedness of continuous landscapes using entropy , 2009, 2009 IEEE Congress on Evolutionary Computation.

[4]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[5]  L. Darrell Whitley,et al.  The dispersion metric and the CMA evolution strategy , 2006, GECCO.

[6]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[7]  Andries Petrus Engelbrecht,et al.  A progressive random walk algorithm for sampling continuous fitness landscapes , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[8]  Andries Petrus Engelbrecht,et al.  Ruggedness, funnels and gradients in fitness landscapes and the effect on PSO performance , 2013, 2013 IEEE Congress on Evolutionary Computation.

[9]  Sudhanshu K. Mishra,et al.  Some New Test Functions for Global Optimization and Performance of Repulsive Particle Swarm Method , 2006 .

[10]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[11]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[12]  Vesselin K. Vassilev,et al.  Smoothness, ruggedness and neutrality of fitness landscapes: from theory to application , 2003 .

[13]  Terry Jones,et al.  Fitness Distance Correlation as a Measure of Problem Difficulty for Genetic Algorithms , 1995, ICGA.

[14]  James Kennedy,et al.  Bare bones particle swarms , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[15]  一将 白髪,et al.  捕食者被食者の関係を導入したHeterogeneous Particle Swarm Optimization , 2012 .

[16]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[17]  Kate Smith-Miles,et al.  Towards insightful algorithm selection for optimisation using meta-learning concepts , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[18]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[19]  Shahryar Rahnamayan,et al.  A novel population initialization method for accelerating evolutionary algorithms , 2007, Comput. Math. Appl..

[20]  Andries Petrus Engelbrecht,et al.  A survey of techniques for characterising fitness landscapes and some possible ways forward , 2013, Inf. Sci..

[21]  Riccardo Poli,et al.  Information landscapes , 2005, GECCO '05.

[22]  Anne Auger,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions , 2009 .

[23]  Katherine M. Malan.  Characterising continuous optimisation problems for particle swarm optimisation performance prediction , 2014 .

[24]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[25]  Sudhanshu K. Mishra,et al.  Performance of Repulsive Particle Swarm Method in Global Optimization of Some Important Test Functions: A Fortran Program , 2006 .

[26]  Thomas Bäck,et al.  Empirical Investigation of Multiparent Recombination Operators in Evolution Strategies , 1997, Evolutionary Computation.

[27]  Mario A. Muñoz,et al.  A Meta-learning Prediction Model of Algorithm Performance for Continuous Optimization Problems , 2012, PPSN.

[28]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[29]  Riccardo Poli,et al.  Information landscapes and problem hardness , 2005, GECCO '05.

[30]  Bernd Bischl,et al.  Algorithm selection based on exploratory landscape analysis and cost-sensitive learning , 2012, GECCO '12.

[31]  Nikolaus Hansen,et al.  Evaluating the CMA Evolution Strategy on Multimodal Test Functions , 2004, PPSN.

[32]  Andries P. Engelbrecht,et al.  Fitness Landscape Analysis for Metaheuristic Performance Prediction , 2014 .

[33]  Sébastien Vérel,et al.  Where are bottlenecks in NK fitness landscapes? , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[34]  James Kennedy,et al.  The particle swarm: social adaptation of knowledge , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).