Stability Analysis of the Particle Swarm Optimization Without Stagnation Assumption

In this letter, we study the first- and second-order stabilities of a stochastic recurrence relation that represents a class of particle swarm optimization (PSO) algorithms. We assume that the personal and global best vectors in that relation are random variables (with arbitrary means and variances) that are updated during the run so that our calculations do not require the stagnation assumption. We prove that the convergence of expectation and variance of the positions generated by that relation is independent of the mean and variance of the distribution of the personal and global best vectors. We also provide convergence boundaries for that relation and compare them with those of standard PSO algorithms (as a specific case of the stochastic recurrence relation) provided in earlier studies.

[1]  Andries Petrus Engelbrecht,et al.  A Convergence Proof for the Particle Swarm Optimiser , 2010, Fundam. Informaticae.

[2]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[3]  Zbigniew Michalewicz,et al.  A locally convergent rotationally invariant particle swarm optimization algorithm , 2014, Swarm Intelligence.

[4]  Andries Petrus Engelbrecht,et al.  A study of particle swarm optimization particle trajectories , 2006, Inf. Sci..

[5]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[6]  M. Clerc,et al.  The swarm and the queen: towards a deterministic and adaptive particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[7]  Peter J. Angeline,et al.  Evolutionary Optimization Versus Particle Swarm Optimization: Philosophy and Performance Differences , 1998, Evolutionary Programming.

[8]  Esperanza García Gonzalo,et al.  Convergence and stochastic stability analysis of particle swarm optimization variants with generic parameter distributions , 2014, Appl. Math. Comput..

[9]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[10]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[11]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[12]  Zbigniew Michalewicz,et al.  An analysis of the velocity updating rule of the particle swarm optimization algorithm , 2014, Journal of Heuristics.

[13]  Zbigniew Michalewicz,et al.  Analysis of Stability, Local Convergence, and Transformation Sensitivity of a Variant of the Particle Swarm Optimization Algorithm , 2016, IEEE Transactions on Evolutionary Computation.

[14]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[15]  Riccardo Poli,et al.  Analysis of the publications on the applications of particle swarm optimisation , 2008 .

[16]  Shiyuan Yang,et al.  Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm , 2007, Inf. Process. Lett..

[17]  Riccardo Poli,et al.  Mean and Variance of the Sampling Distribution of Particle Swarm Optimizers During Stagnation , 2009, IEEE Transactions on Evolutionary Computation.

[18]  Qunfeng Liu,et al.  Order-2 Stability Analysis of Particle Swarm Optimization , 2015, Evolutionary Computation.

[19]  Andries Petrus Engelbrecht,et al.  A generalized theoretical deterministic particle swarm model , 2014, Swarm Intelligence.

[20]  Hyunjoong Kim,et al.  Functional Analysis I , 2017 .

[21]  A. Groenwold,et al.  Comparison of linear and classical velocity update rules in particle swarm optimization: notes on scale and frame invariance , 2007 .

[22]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..