Improving the vector generation strategy of Differential Evolution for large-scale optimization

Several potential weaknesses of DE deteriorate its performance specially when dealing with large-scale problems.Controlling the diversity of trial vectors and exploration capabilities of DE is very important when designing new DE approaches for high-dimensional problems.Two new schemes based on diversifying the trial vectors substantially improve the capabilities of DE for dealing with high-dimensional problems.The new proposals are an alternative way to control the balance between exploration and exploitation in DE.Several state of the art non-hybrid DE schemes are improved by incorporating our proposals. Differential Evolution is an efficient metaheuristic for continuous optimization that suffers from the curse of dimensionality. A large amount of experimentation has allowed researchers to find several potential weaknesses in Differential Evolution. Some of these weaknesses do not significantly affect its performance when dealing with low-dimensional problems, so the research community has not paid much attention to them. The aim of this paper is to provide a better insight into the reasons of the curse of dimensionality and to propose techniques to alleviate this problem. Two different weaknesses are revisited and schemes for dealing with them are devised. The schemes increase the diversity of trial vectors and improve on the exploration capabilities of Differential Evolution. Some important mathematical properties induced by our proposals are studied and compared against those of related schemes. Experimentation with a set of problems with up to 1000 dimensions and with several variants of Differential Evolution shows that the weaknesses analyzed significantly affect the performance of Differential Evolution when used on high-dimensional optimization problems. The gains of the proposals appear when highly exploitative schemes are used. Our proposals allow for high-quality solutions with small populations, meaning that the most significant advantages emerge when dealing with large-scale optimization problems, where the benefits of using small populations have previously been shown.

[1]  Konstantinos E. Parsopoulos,et al.  Cooperative micro-differential evolution for high-dimensional problems , 2009, GECCO.

[2]  Karol R. Opara,et al.  DMEA — An algorithm that combines differential mutation with the fitness proportionate selection , 2011, 2011 IEEE Symposium on Differential Evolution (SDE).

[3]  A. E. Eiben,et al.  Introduction to Evolutionary Computing , 2003, Natural Computing Series.

[4]  Petr Bujok,et al.  Adaptive Variants of Differential Evolution: Towards Control-Parameter-Free Optimizers , 2013, Handbook of Optimization.

[5]  Shahryar Rahnamayan,et al.  Metaheuristics in large-scale global continues optimization: A survey , 2015, Inf. Sci..

[6]  R. Storn,et al.  Differential evolution a simple and efficient adaptive scheme for global optimization over continu , 1997 .

[7]  Xiaodong Li,et al.  Benchmark Functions for the CEC'2010 Special Session and Competition on Large-Scale , 2009 .

[8]  Kenneth A. De Jong,et al.  A Cooperative Coevolutionary Approach to Function Optimization , 1994, PPSN.

[9]  P. N. Suganthan,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[10]  Ponnuthurai N. Suganthan,et al.  Empirical investigations into the exponential crossover of differential evolutions , 2013, Swarm Evol. Comput..

[11]  Carlos A. Coello Coello,et al.  A Study of Multiobjective Metaheuristics When Solving Parameter Scalable Problems , 2010, IEEE Transactions on Evolutionary Computation.

[12]  Janez Brest,et al.  An Analysis of the Control Parameters’ Adaptation in DE , 2008 .

[13]  Antonio LaTorre,et al.  A comprehensive comparison of large scale global optimizers , 2015, Inf. Sci..

[14]  Hongfei Teng,et al.  Cooperative Co-evolutionary Differential Evolution for Function Optimization , 2005, ICNC.

[15]  Zhijian Wu,et al.  Enhanced opposition-based differential evolution for solving high-dimensional continuous optimization problems , 2011, Soft Comput..

[16]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[17]  Xin Yao,et al.  Scalability of generalized adaptive differential evolution for large-scale continuous optimization , 2010, Soft Comput..

[18]  Janez Brest,et al.  Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems , 2006, IEEE Transactions on Evolutionary Computation.

[19]  P. N. Suganthan,et al.  Differential Evolution: A Survey of the State-of-the-Art , 2011, IEEE Transactions on Evolutionary Computation.

[20]  Arturo Hernández Aguirre,et al.  Using Copulas in Estimation of Distribution Algorithms , 2009, MICAI.

[21]  Daniela Zaharie,et al.  Influence of crossover on the behavior of Differential Evolution Algorithms , 2009, Appl. Soft Comput..

[22]  Amit Konar,et al.  Two improved differential evolution schemes for faster global search , 2005, GECCO '05.

[23]  Antonio LaTorre,et al.  A MOS-based dynamic memetic differential evolution algorithm for continuous optimization: a scalability test , 2011, Soft Comput..

[24]  Carlos A. Coello Coello,et al.  On the adaptation of the mutation scale factor in differential evolution , 2015, Optim. Lett..

[25]  A. Kai Qin,et al.  Self-adaptive differential evolution algorithm for numerical optimization , 2005, 2005 IEEE Congress on Evolutionary Computation.

[26]  Carlos A. Coello Coello,et al.  A comparative study of differential evolution variants for global optimization , 2006, GECCO.

[27]  Manuela M. Veloso,et al.  Rational and Convergent Learning in Stochastic Games , 2001, IJCAI.

[28]  Xin Yao,et al.  Large scale evolutionary optimization using cooperative coevolution , 2008, Inf. Sci..

[29]  Janez Brest,et al.  Self-adaptive differential evolution algorithm using population size reduction and three strategies , 2011, Soft Comput..

[30]  I. Good The Bayes/Non-Bayes Compromise: A Brief Review , 1992 .

[31]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[32]  Janez Brest,et al.  Large scale global optimization using self-adaptive differential evolution algorithm , 2010, IEEE Congress on Evolutionary Computation.

[33]  Raymond Chiong,et al.  Evolutionary Optimization: Pitfalls and Booby Traps , 2012, Journal of Computer Science and Technology.

[34]  Ville Tirronen,et al.  Recent advances in differential evolution: a survey and experimental analysis , 2010, Artificial Intelligence Review.

[35]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[36]  Zhijian Wu,et al.  Sequential DE enhanced by neighborhood search for Large Scale Global Optimization , 2010, IEEE Congress on Evolutionary Computation.

[37]  Enrique Alba,et al.  Micro-differential evolution with local search for high dimensional problems , 2013, 2013 IEEE Congress on Evolutionary Computation.

[38]  Andries Petrus Engelbrecht,et al.  Differential evolution in high-dimensional search spaces , 2007, 2007 IEEE Congress on Evolutionary Computation.

[39]  Janez Brest,et al.  Self-adaptive differential evolution algorithm with a small and varying population size , 2012, 2012 IEEE Congress on Evolutionary Computation.

[40]  Riccardo Poli,et al.  Evolving Problems to Learn About Particle Swarm Optimizers and Other Search Algorithms , 2006, IEEE Transactions on Evolutionary Computation.

[41]  Karl-Dirk Kammeyer,et al.  Parameter Study for Differential Evolution Using a Power Allocation Problem Including Interference Cancellation , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[42]  Rainer Laur,et al.  Comparison of Adaptive Approaches for Differential Evolution , 2008, PPSN.

[43]  Yang Tang,et al.  Adaptive population tuning scheme for differential evolution , 2013, Inf. Sci..

[44]  Zhao Yang Dong,et al.  Power system fault diagnosis based on history driven differential evolution and stochastic time domain simulation , 2014, Inf. Sci..

[45]  Janez Brest,et al.  Population size reduction for the differential evolution algorithm , 2008, Applied Intelligence.

[46]  Xiaodong Li,et al.  Cooperative Co-Evolution With Differential Grouping for Large Scale Optimization , 2014, IEEE Transactions on Evolutionary Computation.

[47]  Adam P. Piotrowski,et al.  Adaptive Memetic Differential Evolution with Global and Local neighborhood-based mutation operators , 2013, Inf. Sci..

[48]  S. Nasuto,et al.  Exploration vs exploitation in differential evolution , 2008 .

[49]  Francisco Herrera,et al.  Editorial scalability of evolutionary algorithms and other metaheuristics for large-scale continuous optimization problems , 2011, Soft Comput..

[50]  Sahand Ghavidel,et al.  Modified teaching learning algorithm and double differential evolution algorithm for optimal reactive power dispatch problem: A comparative study , 2014, Inf. Sci..

[51]  H. Abbass The self-adaptive Pareto differential evolution algorithm , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[52]  Antonio LaTorre,et al.  Multiple Offspring Sampling in Large Scale Global Optimization , 2012, 2012 IEEE Congress on Evolutionary Computation.

[53]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[54]  James Montgomery Differential evolution: Difference vectors and movement in solution space , 2009, 2009 IEEE Congress on Evolutionary Computation.

[55]  Hui Li,et al.  Adaptive strategy selection in differential evolution for numerical optimization: An empirical study , 2011, Inf. Sci..

[56]  Ville Tirronen,et al.  Shuffle or update parallel differential evolution for large-scale optimization , 2011, Soft Comput..