CMA-ES: A Function Value Free Second Order Optimization Method

We give a bird's-eye view introduction to the Covariance Matrix Adaptation Evolution Strategy (CMA-ES) and emphasize relevant design aspects of the algorithm, namely its invariance properties. While CMA-ES is gradient and function value free, we show that using the gradient in CMA-ES is possible and can reduce the number of iterations on unimodal, smooth functions.

[1]  Anne Auger,et al.  Principled Design of Continuous Stochastic Search: From Theory to Practice , 2014, Theory and Principled Methods for the Design of Metaheuristics.

[2]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[3]  Nikolaus Hansen,et al.  Injecting External Solutions Into CMA-ES , 2011, ArXiv.

[4]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[5]  Dirk V. Arnold,et al.  Improving Evolution Strategies through Active Covariance Matrix Adaptation , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[6]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.