Bayesian Optimization Approaches for Massively Multi-modal Problems

The optimization of massively multi-modal functions is a challenging task, particularly for problems where the search space can lead the optimization process to local optima. While evolutionary algorithms have been extensively investigated for these optimization problems, Bayesian Optimization algorithms have not been explored to the same extent. In this paper, we study the behavior of Bayesian Optimization as part of a hybrid approach for solving several massively multi-modal functions. We use well-known benchmarks and metrics to evaluate how different variants of Bayesian Optimization deal with multi-modality.

[1]  Alain Pétrowski,et al.  A clearing procedure as a niching method for genetic algorithms , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[2]  Nando de Freitas,et al.  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning , 2010, ArXiv.

[3]  Anatoly Zhigljavsky,et al.  Methods Based on Statistical Models of Multimodal Functions , 2008 .

[4]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[5]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[6]  Yaroslav D. Sergeyev,et al.  Deterministic approaches for solving practical black-box global optimization problems , 2015, Adv. Eng. Softw..

[7]  Mike Preuss,et al.  Niching the CMA-ES via nearest-better clustering , 2010, GECCO '10.

[8]  J. Mockus The Bayesian Approach to Local Optimization , 1989 .

[9]  Hao Wang,et al.  Time complexity reduction in efficient global optimization using cluster kriging , 2017, GECCO.

[10]  Kalyanmoy Deb,et al.  Comparison of multi-modal optimization algorithms based on evolutionary algorithms , 2006, GECCO.

[11]  Xiaodong Yin,et al.  Investigations On Solving the Load Flow Problem By Genetic Algorithms , 1991 .

[12]  C. D. Perttunen,et al.  Lipschitzian optimization without the Lipschitz constant , 1993 .

[13]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[14]  D. Ginsbourger,et al.  A Multi-points Criterion for Deterministic Parallel Global Optimization based on Gaussian Processes , 2008 .

[15]  Bruno Sareni,et al.  Fitness sharing and niching methods revisited , 1998, IEEE Trans. Evol. Comput..

[16]  Xiaodong Li,et al.  Benchmark Functions for CEC'2013 Special Session and Competition on Niching Methods for Multimodal Function Optimization' , 2013 .

[17]  Kwong-Sak Leung,et al.  Protein structure prediction on a lattice model via multimodal optimization techniques , 2010, GECCO '10.

[18]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[19]  Cheng Li,et al.  A Divide-and-Conquer Bayesian Approach to Large-Scale Kriging , 2017, 1712.09767.

[20]  Jonathan E. Fieldsend,et al.  Running Up Those Hills: Multi-modal search with the niching migratory multi-swarm optimiser , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[21]  Neil D. Lawrence,et al.  Batch Bayesian Optimization via Local Penalization , 2015, AISTATS.

[22]  M. J. D. Powell,et al.  An efficient method for finding the minimum of a function of several variables without calculating derivatives , 1964, Comput. J..

[23]  B. Silverman,et al.  Some Aspects of the Spline Smoothing Approach to Non‐Parametric Regression Curve Fitting , 1985 .