Orthogonal Learning Particle Swarm Optimization

Particle swarm optimization (PSO) relies on its learning strategy to guide its search direction. Traditionally, each particle utilizes its historical best experience and its neighborhood's best experience through linear summation. Such a learning strategy is easy to use, but is inefficient when searching in complex problem spaces. Hence, designing learning strategies that can utilize previous search information (experience) more efficiently has become one of the most salient and active PSO research topics. In this paper, we proposes an orthogonal learning (OL) strategy for PSO to discover more useful information that lies in the above two experiences via orthogonal experimental design. We name this PSO as orthogonal learning particle swarm optimization (OLPSO). The OL strategy can guide particles to fly in better directions by constructing a much promising and efficient exemplar. The OL strategy can be applied to PSO with any topological structure. In this paper, it is applied to both global and local versions of PSO, yielding the OLPSO-G and OLPSO-L algorithms, respectively. This new learning strategy and the new algorithms are tested on a set of 16 benchmark functions, and are compared with other PSO algorithms and some state of the art evolutionary algorithms. The experimental results illustrate the effectiveness and efficiency of the proposed learning strategy and algorithms. The comparisons show that OLPSO significantly improves the performance of PSO, offering faster global convergence, higher solution quality, and stronger robustness.

[1]  Jenn-Long Liu,et al.  Novel orthogonal momentum-type particle swarm optimization applied to solve large parameter optimization problems , 2008 .

[2]  Shinn-Ying Ho,et al.  An orthogonal simulated annealing algorithm for large floorplanning problems , 2004, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[3]  Frans van den Bergh,et al.  A NICHING PARTICLE SWARM OPTIMIZER , 2002 .

[4]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[5]  Michael N. Vrahatis,et al.  On the computation of all global minimizers through particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[6]  Salman Mohagheghi,et al.  Particle Swarm Optimization: Basic Concepts, Variants and Applications in Power Systems , 2008, IEEE Transactions on Evolutionary Computation.

[7]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[8]  Yun Shang,et al.  A Note on the Extended Rosenbrock Function , 2006 .

[9]  M. Clerc,et al.  The swarm and the queen: towards a deterministic and adaptive particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[10]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[11]  Thomas Bäck,et al.  Evolutionary computation: Toward a new philosophy of machine intelligence , 1997, Complex..

[12]  Andries Petrus Engelbrecht,et al.  Particle swarm optimization: an introduction and its recent developments , 2007, Annual Conference on Genetic and Evolutionary Computation.

[13]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[14]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[15]  Paul S. Andrews,et al.  An Investigation into Mutation Operators for Particle Swarm Optimization , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[16]  Suganthan [IEEE 1999. Congress on Evolutionary Computation-CEC99 - Washington, DC, USA (6-9 July 1999)] Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406) - Particle swarm optimiser with neighbourhood operator , 1999 .

[17]  张军,et al.  Orthogonal Methods Based Ant Colony Search for Solving Continuous Optimization Problems , 2008 .

[18]  Shinn-Ying Ho,et al.  OPSO: Orthogonal Particle Swarm Optimization and Its Application to Task Assignment Problems , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[19]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[20]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[21]  James Kennedy,et al.  Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[22]  Karen A. F. Copeland Design and Analysis of Experiments, 5th Ed. , 2001 .

[23]  Margaret J. Robertson,et al.  Design and Analysis of Experiments , 2006, Handbook of statistics.

[24]  Yuping Wang,et al.  An orthogonal genetic algorithm with quantization for global numerical optimization , 2001, IEEE Trans. Evol. Comput..

[25]  Thomas Kiel Rasmussen,et al.  Hybrid Particle Swarm Optimiser with breeding and subpopulations , 2001 .

[26]  Jun Zhang,et al.  An Enhanced Genetic Algorithm with Orthogonal Design , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[27]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[28]  Russell C. Eberhart,et al.  Multiobjective optimization using dynamic neighborhood particle swarm optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[29]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[30]  Jing J. Liang,et al.  Dynamic multi-swarm particle swarm optimizer , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[31]  J. Kennedy Stereotyping: improving particle swarm performance with cluster analysis , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[32]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .

[33]  Russell C. Eberhart,et al.  Recent advances in particle swarm , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[34]  Andries Petrus Engelbrecht,et al.  Locating multiple optima using particle swarm optimization , 2007, Appl. Math. Comput..

[35]  Anne Auger,et al.  Performance evaluation of an advanced local search evolutionary algorithm , 2005, 2005 IEEE Congress on Evolutionary Computation.

[36]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[37]  P. J. Angeline,et al.  Using selection to improve particle swarm optimization , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[38]  Jacek M. Zurada,et al.  An approach to multimodal biomedical image registration utilizing particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[39]  S. N. Sivanandam,et al.  Dynamic Task Scheduling with Load Balancing using Hybrid Particle Swarm Optimization , 2009 .

[40]  Yuhui Shi,et al.  Particle swarm optimization: developments, applications and resources , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[41]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[42]  Kalyan Veeramachaneni,et al.  Fitness-distance-ratio based particle swarm optimization , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[43]  Rui Mendes,et al.  Neighborhood topologies in fully informed and best-of-neighborhood particle swarms , 2006 .

[44]  Qingfu Zhang,et al.  An orthogonal genetic algorithm for multimedia multicast routing , 1999, IEEE Trans. Evol. Comput..

[45]  Kalyanmoy Deb,et al.  A Computationally Efficient Evolutionary Algorithm for Real-Parameter Optimization , 2002, Evolutionary Computation.

[46]  Chia-Feng Juang,et al.  A hybrid of genetic algorithm and particle swarm optimization for recurrent network design , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[47]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[48]  J. Ford,et al.  Hybrid estimation of distribution algorithm for global optimization , 2004 .

[49]  Carlos A. Coello Coello,et al.  Handling multiple objectives with particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[50]  Shinn-Ying Ho,et al.  OSA: orthogonal simulated annealing algorithm and its application to designing mixed H2/H∞ optimal controllers , 2004, IEEE Trans. Syst. Man Cybern. Part A.

[51]  Andries Petrus Engelbrecht,et al.  Particle swarm optimization approaches to coevolve strategies for the iterated prisoner's dilemma , 2005, IEEE Transactions on Evolutionary Computation.

[52]  Shinn-Ying Ho,et al.  Intelligent evolutionary algorithms for large parameter optimization problems , 2004, IEEE Trans. Evol. Comput..

[53]  Xiaodong Li,et al.  This article has been accepted for inclusion in a future issue. IEEE TRANSACTIONS ON EVOLUTIONARY COMPUTATION 1 Locating and Tracking Multiple Dynamic Optima by a Particle Swarm Model Using Speciation , 2022 .

[54]  A.P. Engelbrecht,et al.  Learning to play games using a PSO-based competitive learning approach , 2004, IEEE Transactions on Evolutionary Computation.

[55]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[56]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[57]  Russell C. Eberhart,et al.  Comparison between Genetic Algorithms and Particle Swarm Optimization , 1998, Evolutionary Programming.

[58]  J. Kennedy,et al.  Neighborhood topologies in fully informed and best-of-neighborhood particle swarms , 2003, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[59]  Jun Zhang,et al.  Adaptive Particle Swarm Optimization , 2008, ANTS Conference.