Particle Swarm Optimization and Differential Evolution Algorithms: Technical Analysis, Applications and Hybridization Perspectives

Since the beginning of the nineteenth century, a significant evolution in optimization theory has been noticed. Classical linear programming and traditional non-linear optimization techniques such as Lagrange’s Multiplier, Bellman’s principle and Pontyagrin’s principle were prevalent until this century. Unfortunately, these derivative based optimization techniques can no longer be used to determine the optima on rough non-linear surfaces. One solution to this problem has already been put forward by the evolutionary algorithms research community. Genetic algorithm (GA), enunciated by Holland, is one such popular algorithm. This chapter provides two recent algorithms for evolutionary optimization – well known as particle swarm optimization (PSO) and differential evolution (DE). The algorithms are inspired by biological and sociological motivations and can take care of optimality on rough, discontinuous and multimodal surfaces. The chapter explores several schemes for controlling the convergence behaviors of PSO and DE by a judicious selection of their parameters. Special emphasis is given on the hybridizations of PSO and DE algorithms with other soft computing tools. The article finally discusses the mutual synergy of PSO with DE leading to a more powerful global search algorithm and its practical applications.

[1]  Amit Konar,et al.  A swarm intelligence approach to the synthesis of two-dimensional IIR filters , 2007, Eng. Appl. Artif. Intell..

[2]  M. A. Abido.  Optimal des'ign of Power System Stabilizers Using Particle Swarm Opt'imization , 2002, IEEE Power Engineering Review.

[3]  Peter J. Angeline,et al.  Evolutionary Optimization Versus Particle Swarm Optimization: Philosophy and Performance Differences , 1998, Evolutionary Programming.

[4]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[5]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[6]  Zhao Hui,et al.  Optimal Design of Power System Stabilizer Using Particle Swarm Optimization , 2006 .

[7]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[8]  Jacek M. Zurada,et al.  An approach to multimodal biomedical image registration utilizing particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[9]  T. Krink,et al.  Particle swarm optimisation with spatial particle extension , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[10]  W. Land,et al.  A new training algorithm for the general regression neural network , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[11]  Amit Konar,et al.  Two-Dimensional IIR Filter Design with Modern Search Heuristics: a Comparative Study , 2006, Int. J. Comput. Intell. Appl..

[12]  T. Krink,et al.  Extending particle swarm optimisers with self-organized criticality , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[13]  Sandra Paterlini,et al.  Differential evolution and particle swarm optimisation in partitional clustering , 2006, Comput. Stat. Data Anal..

[14]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[15]  James Kennedy,et al.  Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[16]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[17]  A.P. Engelbrecht,et al.  Learning to play games using a PSO-based competitive learning approach , 2004, IEEE Transactions on Evolutionary Computation.

[18]  Russell C. Eberhart,et al.  Human tremor analysis using particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[19]  Yoshikazu Fukuyama,et al.  A particle swarm optimization for reactive power and voltage control considering voltage security assessment , 2000 .

[20]  Ajith Abraham,et al.  Chaotic dynamic characteristics in swarm intelligence , 2007, Appl. Soft Comput..

[21]  Marcus Randall,et al.  A survey of ant colony and particle swarm meta-heuristics and their application to discrete optimisation problems , 2001 .

[22]  Y. Rahmat-Samii,et al.  Particle swarm, genetic algorithm, and their hybrids: optimization of a profiled corrugated horn antenna , 2002, IEEE Antennas and Propagation Society International Symposium (IEEE Cat. No.02CH37313).

[23]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[24]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[25]  Andries P. Engelbrecht,et al.  Effects of swarm size on Cooperative Particle Swarm Optimisers , 2001 .

[26]  Amit Konar,et al.  Kernel based automatic clustering using modified particle swarm optimization algorithm , 2007, GECCO '07.

[27]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[28]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[29]  R. Eberhart,et al.  Fuzzy adaptive particle swarm optimization , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[30]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[31]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[32]  M.N.S. Swamy,et al.  Design of two-dimensional recursive filters using genetic algorithms , 2003 .

[33]  R. W. Derksen,et al.  Differential Evolution in Aerodynamic Optimization , 1999 .

[34]  Feng-Sheng Wang,et al.  Multiobjective parameter estimation problems of fermentation processes using a high ethanol tolerance yeast , 2000 .

[35]  Yuhui Shi,et al.  Extracting rules from fuzzy neural network by particle swarm optimisation , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[36]  Michael N. Vrahatis,et al.  On the computation of all global minimizers through particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[37]  Jan Peters,et al.  Computational Intelligence: Principles, Techniques and Applications , 2007, Comput. J..

[38]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[39]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[40]  Thiemo Krink,et al.  The LifeCycle Model: Combining Particle Swarm Optimisation, Genetic Algorithms and HillClimbers , 2002, PPSN.

[41]  Kay Hameyer,et al.  Optimization of radial active magnetic bearings using the finite element technique and the differential evolution algorithm , 2000 .

[42]  Jouni Lampinen,et al.  A Trigonometric Mutation Operation to Differential Evolution , 2003, J. Glob. Optim..

[43]  J. Salerno,et al.  Using the particle swarm optimization technique to train a recurrent neural model , 1997, Proceedings Ninth IEEE International Conference on Tools with Artificial Intelligence.

[44]  Vladimiro Miranda,et al.  EPSO-evolutionary particle swarm optimization, a new algorithm with applications in power systems , 2002, IEEE/PES Transmission and Distribution Conference and Exhibition.

[45]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[46]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[47]  A. Hamler,et al.  Analysis of iron loss in interior permanent magnet synchronous motor over a wide-speed range of constant output power operation , 2000 .

[48]  Ajith Abraham,et al.  Variable Neighborhood Particle Swarm Optimization Algorithm , 2006 .

[49]  Peter J. Bentley,et al.  Improvised music with swarms , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[50]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[51]  Frans van den Bergh,et al.  Particle Swarm Weight Initialization In Multi-Layer Perceptron Artificial Neural Networks , 1999 .

[52]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[53]  Steven Doyle,et al.  Automated mirror design using an evolution strategy , 1999 .

[54]  Amit Konar,et al.  Two improved differential evolution schemes for faster global search , 2005, GECCO '05.

[55]  Thomas Kiel Rasmussen,et al.  Hybrid Particle Swarm Optimiser with breeding and subpopulations , 2001 .