A dynamic neighborhood learning based particle swarm optimizer for global numerical optimization

The concept of particle swarms originated from the simulation of the social behavior commonly observed in animal kingdom and evolved into a very simple but efficient technique for optimization in recent past. Since its advent in 1995, the Particle Swarm Optimization (PSO) algorithm has attracted the attention of a lot of researchers all over the world resulting into a huge number of variants of the basic algorithm as well as many parameter selection/control strategies. PSO relies on the learning strategy of the individuals to guide its search direction. Traditionally, each particle utilizes its historical best experience as well as the global best experience of the whole swarm through linear summation. The Comprehensive Learning PSO (CLPSO) was proposed as a powerful variant of PSO that enhances the diversity of the population by encouraging each particle to learn from different particles on different dimensions, in the metaphor that the best particle, despite having the highest fitness, does not always offer a better value in every dimension. This paper presents a variant of single-objective PSO called Dynamic Neighborhood Learning Particle Swarm Optimizer (DNLPSO), which uses learning strategy whereby all other particles' historical best information is used to update a particle's velocity as in CLPSO. But in contrast to CLPSO, in DNLPSO, the exemplar particle is selected from a neighborhood. This strategy enables the learner particle to learn from the historical information of its neighborhood or sometimes from that of its own. Moreover, the neighborhoods are made dynamic in nature i.e. they are reformed after certain intervals. This helps the diversity of the swarm to be preserved in order to discourage premature convergence. Experiments were conducted on 16 numerical benchmarks in 10, 30 and 50 dimensions, a set of five constrained benchmarks and also on a practical engineering optimization problem concerning the spread-spectrum radar poly-phase code design. The results demonstrate very competitive performance of DNLPSO while locating the global optimum on complicated and multimodal fitness landscapes when compared with five other recent variants of PSO.

[1]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[2]  N. P. Padhy,et al.  Application of particle swarm optimization technique and its variants to generation expansion planning problem , 2004 .

[3]  Russell C. Eberhart,et al.  Guest Editorial Special Issue on Particle Swarm Optimization , 2004, IEEE Trans. Evol. Comput..

[4]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[5]  Salman Mohagheghi,et al.  Particle Swarm Optimization: Basic Concepts, Variants and Applications in Power Systems , 2008, IEEE Transactions on Evolutionary Computation.

[6]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[7]  Fred W. Glover,et al.  Tabu Search , 1997, Handbook of Heuristics.

[8]  Andries Petrus Engelbrecht,et al.  Particle swarm optimization: an introduction and its recent developments , 2007, Annual Conference on Genetic and Evolutionary Computation.

[9]  Konstantinos E. Parsopoulos,et al.  UPSO: A Unified Particle Swarm Optimization Scheme , 2019, International Conference of Computational Methods in Sciences and Engineering 2004 (ICCMSE 2004).

[10]  Chukwudi Anyakoha,et al.  A review of particle swarm optimization. Part II: hybridisation, combinatorial, multicriteria and constrained optimization, and indicative applications , 2008, Natural Computing.

[11]  Xiao-Feng Xie,et al.  DEPSO: hybrid particle swarm with differential evolution operator , 2003, SMC'03 Conference Proceedings. 2003 IEEE International Conference on Systems, Man and Cybernetics. Conference Theme - System Security and Assurance (Cat. No.03CH37483).

[12]  Rainer Laur,et al.  RUN TIME ANALYSIS REGARDING STOPPING CRITERIA FOR DIFFERENTIAL EVOLUTION AND PARTICLE SWARM OPTIMIZATION , 2005 .

[13]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[14]  Chukwudi Anyakoha,et al.  A review of particle swarm optimization. Part I: background and development , 2007, Natural Computing.

[15]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[16]  Jun Zhang,et al.  Orthogonal Learning Particle Swarm Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[17]  J. Kennedy,et al.  Neighborhood topologies in fully informed and best-of-neighborhood particle swarms , 2003, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[18]  R. Venkata Rao,et al.  Teaching-Learning-Based Optimization: An optimization method for continuous non-linear large scale problems , 2012, Inf. Sci..

[19]  Francisco Herrera,et al.  Genetics-Based Machine Learning for Rule Induction: State of the Art, Taxonomy, and Comparative Study , 2010, IEEE Transactions on Evolutionary Computation.

[20]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[21]  James Kennedy,et al.  Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[22]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[23]  Mirjana Cangalovic,et al.  Solving spread spectrum radar polyphase code design problem by tabu search and variable neighbourhood search , 2003, Eur. J. Oper. Res..

[24]  Jing J. Liang,et al.  Dynamic multi-swarm particle swarm optimizer , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[25]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[26]  Vladimiro Miranda,et al.  EPSO - best-of-two-worlds meta-heuristic applied to power system problems , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[27]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[28]  Jing J. Liang,et al.  Novel composition test functions for numerical global optimization , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[29]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[30]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[31]  Alessandro Giua,et al.  Guest Editorial , 2001, Discrete event dynamic systems.

[32]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[33]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[34]  Ali R. Yildiz,et al.  Hybrid Taguchi-Harmony Search Algorithm for Solving Engineering Optimization Problems , 2008 .

[35]  Miroslav L. Dukic,et al.  A Method of a Spread-Spectrum Radar Polyphase Code Design , 1990, IEEE J. Sel. Areas Commun..

[36]  Ali R. Yildiz,et al.  A novel hybrid immune algorithm for global optimization in design and manufacturing , 2009 .

[37]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[38]  Riccardo Poli,et al.  Dynamics and stability of the sampling distribution of particle swarm optimisers via moment analysis , 2008 .

[39]  R. Venkata Rao,et al.  Teaching-learning-based optimization: A novel method for constrained mechanical design optimization problems , 2011, Comput. Aided Des..

[40]  Yoshikazu Fukuyama,et al.  A hybrid particle swarm optimization for distribution state estimation , 2003, 2003 IEEE Power Engineering Society General Meeting (IEEE Cat. No.03CH37491).

[41]  Ali Rıza Yıldız,et al.  A novel particle swarm optimization approach for product design and manufacturing , 2008 .

[42]  Kazuhiro Saitou,et al.  Topology Synthesis of Multicomponent Structural Assemblies in Continuum Domains , 2011 .

[43]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[44]  Chenggong Zhang,et al.  Scale-free fully informed particle swarm optimization algorithm , 2011, Inf. Sci..

[45]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[46]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[47]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[48]  Mohamed E. El-Hawary,et al.  A Survey of Particle Swarm Optimization Applications in Electric Power Systems , 2009, IEEE Transactions on Evolutionary Computation.