Multimodal Optimization by Covariance Matrix Self-Adaptation Evolution Strategy with Repelling Subpopulations

During the recent decades, many niching methods have been proposed and empirically verified on some available test problems. They often rely on some particular assumptions associated with the distribution, shape, and size of the basins, which can seldom be made in practical optimization problems. This study utilizes several existing concepts and techniques, such as taboo points, normalized Mahalanobis distance, and the Ursem’s hill-valley function in order to develop a new tool for multimodal optimization, which does not make any of these assumptions. In the proposed method, several subpopulations explore the search space in parallel. Offspring of a subpopulation are forced to maintain a sufficient distance to the center of fitter subpopulations and the previously identified basins, which are marked as taboo points. The taboo points repel the subpopulation to prevent convergence to the same basin. A strategy to update the repelling power of the taboo points is proposed to address the challenge of basins of dissimilar size. The local shape of a basin is also approximated by the distribution of the subpopulation members converging to that basin. The proposed niching strategy is incorporated into the covariance matrix self-adaptation evolution strategy (CMSA-ES), a potent global optimization method. The resultant method, called the covariance matrix self-adaptation with repelling subpopulations (RS-CMSA), is assessed and compared to several state-of-the-art niching methods on a standard test suite for multimodal optimization. An organized procedure for parameter setting is followed which assumes a rough estimation of the desired/expected number of minima available. Performance sensitivity to the accuracy of this estimation is also studied by introducing the concept of robust mean peak ratio. Based on the numerical results using the available and the introduced performance measures, RS-CMSA emerges as the most successful method when robustness and efficiency are considered at the same time.

[1]  Ali Ahrari,et al.  An improved evolution strategy with adaptive population size , 2015 .

[2]  Marek Kisiel-Dorohinicki,et al.  Maintaining Population Diversity in Evolution Strategy for Engineering Problems , 2008, IEA/AIE.

[3]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[4]  R. K. Ursem Multi-objective Optimization using Evolutionary Algorithms , 2009 .

[5]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[6]  Günter Rudolph,et al.  When parameter tuning actually is parameter control , 2011, GECCO '11.

[7]  Bernhard Sendhoff,et al.  Covariance Matrix Adaptation Revisited - The CMSA Evolution Strategy - , 2008, PPSN.

[8]  Kalyanmoy Deb,et al.  Simultaneous topology, shape and size optimization of truss structures by fully stressed design based on evolution strategy , 2015 .

[9]  Kerstin Vogler,et al.  Applications Of Multi Objective Evolutionary Algorithms , 2016 .

[10]  Nikolaus Hansen,et al.  Evaluating the CMA Evolution Strategy on Multimodal Test Functions , 2004, PPSN.

[11]  Xiaodong Li,et al.  Niching Without Niching Parameters: Particle Swarm Optimization Using a Ring Topology , 2010, IEEE Transactions on Evolutionary Computation.

[12]  Jonathan Goldstein,et al.  When Is ''Nearest Neighbor'' Meaningful? , 1999, ICDT.

[13]  Samir W. Mahfoud Niching methods for genetic algorithms , 1996 .

[14]  R. K. Ursem Multinational evolutionary algorithms , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[15]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[16]  Ofer M. Shir,et al.  Adaptive Niche Radii and Niche Shapes Approaches for Niching with the CMA-ES , 2010, Evolutionary Computation.

[17]  Ponnuthurai N. Suganthan,et al.  Real-parameter evolutionary multimodal optimization - A survey of the state-of-the-art , 2011, Swarm Evol. Comput..

[18]  Jonathan E. Fieldsend,et al.  Running Up Those Hills: Multi-modal search with the niching migratory multi-swarm optimiser , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[19]  M. Shariat Panahi,et al.  GEM: A novel evolutionary optimization method with improved neighborhood search , 2009, Appl. Math. Comput..

[20]  Bernhard Sendhoff,et al.  Robust Optimization - A Comprehensive Survey , 2007 .

[21]  Dumitru Dumitrescu,et al.  Disburdening the species conservation evolutionary algorithm of arguing with radii , 2007, GECCO '07.

[22]  Minqiang Li,et al.  A hybrid niching PSO enhanced with recombination-replacement crowding strategy for multimodal function optimization , 2012, Appl. Soft Comput..

[23]  Kalyanmoy Deb,et al.  A parameterless-niching-assisted bi-objective approach to multimodal optimization , 2013, 2013 IEEE Congress on Evolutionary Computation.

[24]  Nikolaus Hansen,et al.  A restart CMA evolution strategy with increasing population size , 2005, 2005 IEEE Congress on Evolutionary Computation.

[25]  Ole J Mengshoel,et al.  The Crowding Approach to Niching in Genetic Algorithms , 2008, Evolutionary Computation.

[26]  Kalyanmoy Deb,et al.  Multimodal Optimization Using a Bi-Objective Evolutionary Algorithm , 2012, Evolutionary Computation.

[27]  Jie Yao,et al.  Bi-Objective Multipopulation Genetic Algorithm for Multimodal Function Optimization , 2010, IEEE Transactions on Evolutionary Computation.

[28]  Thomas Bäck,et al.  Contemporary Evolution Strategies , 2013, Natural Computing Series.

[29]  Ali Ahrari,et al.  Grenade Explosion Method - A novel tool for optimization of multimodal functions , 2010, Appl. Soft Comput..

[30]  Fred W. Glover,et al.  Future paths for integer programming and links to artificial intelligence , 1986, Comput. Oper. Res..

[31]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[32]  Xiaodong Li,et al.  Benchmark Functions for CEC'2013 Special Session and Competition on Niching Methods for Multimodal Function Optimization' , 2013 .

[33]  Ponnuthurai N. Suganthan,et al.  A Distance-Based Locally Informed Particle Swarm Model for Multimodal Optimization , 2013, IEEE Transactions on Evolutionary Computation.

[34]  Oliver Kramer,et al.  Finite life span for improving the selection scheme in evolution strategies , 2017, Soft Comput..

[35]  Kalyanmoy Deb,et al.  An Investigation of Niche and Species Formation in Genetic Function Optimization , 1989, ICGA.

[36]  Swagatam Das,et al.  An Improved Parent-Centric Mutation With Normalized Neighborhoods for Inducing Niching Behavior in Differential Evolution , 2014, IEEE Transactions on Cybernetics.

[37]  Jing J. Liang,et al.  Differential Evolution With Neighborhood Mutation for Multimodal Optimization , 2012, IEEE Transactions on Evolutionary Computation.

[38]  Kwong-Sak Leung,et al.  Genetic Algorithm with adaptive elitist-population strategies for multimodal function optimization , 2011, Appl. Soft Comput..

[39]  Anne Auger,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions , 2009 .

[40]  Swagatam Das,et al.  Multimodal optimization by artificial weed colonies enhanced with localized group search optimizers , 2013, Appl. Soft Comput..

[41]  Jun Chen,et al.  The Shifting Balance Genetic Algorithm as More than Just Another Island Model GA , 2004, GECCO.

[42]  Kalyanmoy Deb,et al.  Comparison of multi-modal optimization algorithms based on evolutionary algorithms , 2006, GECCO.

[43]  P. John Clarkson,et al.  A Species Conserving Genetic Algorithm for Multimodal Function Optimization , 2002, Evolutionary Computation.

[44]  Ponnuthurai N. Suganthan,et al.  Novel multimodal problems and differential evolution with ensemble of restricted tournament selection , 2010, IEEE Congress on Evolutionary Computation.

[45]  Shengxiang Yang,et al.  Force-imitated particle swarm optimization using the near-neighbor effect for locating multiple optima , 2012, Inf. Sci..

[46]  Dumitru Dumitrescu,et al.  Multimodal Optimization by Means of a Topological Species Conservation Algorithm , 2010, IEEE Transactions on Evolutionary Computation.

[47]  Ofer M. Shir,et al.  Niche Radius Adaptation with Asymmetric Sharing , 2008, PPSN.

[48]  Andries Petrus Engelbrecht,et al.  A novel particle swarm niching technique based on extensive vector operations , 2010, Natural Computing.

[49]  Ofer M. Shir,et al.  Performance analysis of niching algorithms based on derandomized-ES variants , 2007, GECCO '07.

[50]  Nikolaus Hansen,et al.  Benchmarking a BI-population CMA-ES on the BBOB-2009 function testbed , 2009, GECCO '09.

[51]  Michael G. Epitropakis,et al.  Results of the 2013 IEEE CEC Competition on Niching Methods for Multimodal Optimization , 2013 .

[52]  Mike Preuss,et al.  Improved Topological Niching for Real-Valued Global Optimization , 2012, EvoApplications.

[53]  Fabio Freschi,et al.  Comparison of artificial immune systems and genetic algorithms in electrical engineering optimization , 2006 .

[54]  Alain Pétrowski,et al.  A clearing procedure as a niching method for genetic algorithms , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[55]  Xin Yao,et al.  Fast Evolution Strategies , 1997, Evolutionary Programming.

[56]  P. Siarry,et al.  FITTING OF TABU SEARCH TO OPTIMIZE FUNCTIONS OF CONTINUOUS VARIABLES , 1997 .

[57]  L.N. de Castro,et al.  An artificial immune network for multimodal function optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[58]  Ralph R. Martin,et al.  A Sequential Niche Technique for Multimodal Function Optimization , 1993, Evolutionary Computation.

[59]  Hans-Paul Schwefel,et al.  Evolution and Optimum Seeking: The Sixth Generation , 1993 .

[60]  Kay Chen Tan,et al.  Multimodal Optimization Using a Biobjective Differential Evolution Algorithm Enhanced With Mean Distance-Based Selection , 2013, IEEE Transactions on Evolutionary Computation.

[61]  Bruno Sareni,et al.  Fitness sharing and niching methods revisited , 1998, IEEE Trans. Evol. Comput..

[62]  Ofer M. Shir,et al.  Niching with derandomized evolution strategies in artificial and real-world landscapes , 2009, Natural Computing.

[63]  Ingo Rechenberg,et al.  Case studies in evolutionary experimentation and computation , 2000 .

[64]  Shengxiang Yang,et al.  A memetic particle swarm optimization algorithm for multimodal optimization problems , 2011, 2011 Chinese Control and Decision Conference (CCDC).

[65]  Mike Preuss,et al.  Niching the CMA-ES via nearest-better clustering , 2010, GECCO '10.

[66]  Günter Rudolph,et al.  Contemporary Evolution Strategies , 1995, ECAL.

[67]  Michael G. Epitropakis,et al.  Finding multiple global optima exploiting differential evolution's niching capability , 2011, 2011 IEEE Symposium on Differential Evolution (SDE).