SPSO 2011: analysis of stability; local convergence; and rotation sensitivity

In a particle swarm optimization algorithm (PSO) it is essential to guarantee convergence of particles to a point in the search space (this property is called stability of particles). It is also important that the PSO algorithm converges to a local optimum (this is called the local convergence property). Further, it is usually expected that the performance of the PSO algorithm is not affected by rotating the search space (this property is called the rotation sensitivity). In this paper, these three properties, i.e. stability of particles, local convergence, and rotation sensitivity are investigated for a variant of PSO called Standard PSO2011 (SPSO2011). We experimentally define boundaries for the parameters of this algorithm in such a way that if the parameters are selected in these boundaries, the particles are stable, i.e. particles converge to a point in the search space. Also, we show that, unlike earlier versions of PSO, these boundaries are dependent on the number of dimensions of the problem. Moreover, we show that the algorithm is not locally convergent in general case. Finally, we provide a proof and experimental evidence that the algorithm is rotation invariant.

[1]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[2]  M. Jiang,et al.  Particle Swarm Optimization - Stochastic Trajectory Analysis and Parameter Selection , 2007 .

[3]  R. Salomon.  Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms. , 1996, Bio Systems.

[4]  James Kennedy,et al.  Defining a Standard for Particle Swarm Optimization , 2007, 2007 IEEE Swarm Intelligence Symposium.

[5]  Anne Auger,et al.  Empirical comparisons of several derivative free optimization algorithms , 2009 .

[6]  N. Baba.  Convergence of a random optimization method for constrained optimization problems , 1981 .

[7]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[8]  A. Groenwold,et al.  Comparison of linear and classical velocity update rules in particle swarm optimization: notes on scale and frame invariance , 2007 .

[9]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[10]  Xiang Li,et al.  A hybrid particle swarm with velocity mutation for constraint optimization problems , 2013, GECCO '13.

[11]  Riccardo Poli,et al.  Analysis of the publications on the applications of particle swarm optimisation , 2008 .

[12]  Maurice Clerc,et al.  Standard Particle Swarm Optimisation , 2012 .

[13]  Mauricio Zambrano-Bigiarini,et al.  Standard Particle Swarm Optimisation 2011 at CEC-2013: A baseline for future PSO improvements , 2013, 2013 IEEE Congress on Evolutionary Computation.

[14]  N. Hansen,et al.  PSO Facing Non-Separable and Ill-Conditioned Problems , 2008 .

[15]  M. Clerc,et al.  Particle Swarm Optimization , 2006 .

[16]  Andries Petrus Engelbrecht,et al.  A Convergence Proof for the Particle Swarm Optimiser , 2010, Fundam. Informaticae.

[17]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[18]  Thomas Stützle,et al.  Frankenstein's PSO: A Composite Particle Swarm Optimization Algorithm , 2009, IEEE Transactions on Evolutionary Computation.

[19]  Andries Petrus Engelbrecht,et al.  A study of particle swarm optimization particle trajectories , 2006, Inf. Sci..

[20]  Per Kristian Lehre,et al.  Finite First Hitting Time versus Stochastic Convergence in Particle Swarm Optimisation , 2011, ArXiv.

[21]  Manoj Kumar Tiwari,et al.  Swarm Intelligence, Focus on Ant and Particle Swarm Optimization , 2007 .

[22]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[23]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[24]  William M. Spears,et al.  Biases in Particle Swarm Optimization , 2010, Int. J. Swarm Intell. Res..

[25]  Dennis Spellman,et al.  Vector analysis and an introduction to tensor analysis , 1980 .

[26]  Shiyuan Yang,et al.  Stochastic convergence analysis and parameter selection of the standard particle swarm optimization algorithm , 2007, Inf. Process. Lett..

[27]  Riccardo Poli,et al.  Mean and Variance of the Sampling Distribution of Particle Swarm Optimizers During Stagnation , 2009, IEEE Transactions on Evolutionary Computation.

[28]  Qinghai Bai,et al.  Analysis of Particle Swarm Optimization Algorithm , 2010, Comput. Inf. Sci..

[29]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[30]  Rolf Wanka,et al.  Particle swarm optimization almost surely finds local optima , 2013, GECCO '13.

[31]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[32]  Roger J.-B. Wets,et al.  Minimization by Random Search Techniques , 1981, Math. Oper. Res..