Principled Design of Continuous Stochastic Search: From Theory to Practice

We derive a stochastic search procedure for parameter optimization from two first principles: (1) imposing the least prior assumptions, namely by maximum entropy sampling, unbiasedness and invariance; (2) exploiting all available information under the constraints imposed by (1). We additionally require that two of the most basic functions can be solved reasonably fast. Given these principles, two principal heuristics are used: reinforcing of good solutions and good steps (increasing their likelihood) and rendering successive steps orthogonal. The resulting search algorithm is the covariance matrix adaptation evolution strategy, CMA-ES, that coincides to a great extent to a natural gradient descent. The invariance properties of the CMA-ES are formalized, as are its maximum likelihood and stationarity properties. A small parameter study for a specific heuristic—deduced from the principles of reinforcing good steps and exploiting all information—is presented, namely for the cumulation of an evolution or search path. Experiments on two noisy functions are provided.

[1]  Nikolaus Hansen,et al.  CMA-ES with Two-Point Step-Size Adaptation , 2008, ArXiv.

[2]  N. Hansen,et al.  Sizing the population with respect to the local progress in (1,/spl lambda/)-evolution strategies-a theoretical analysis , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[3]  Anne Auger,et al.  Experimental Comparisons of Derivative Free Optimization Algorithms , 2009, SEA.

[4]  Hans-Paul Schwefel,et al.  Numerical optimization of computer models , 1981 .

[5]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[6]  Anne Auger,et al.  Comparing results of 31 algorithms from the black-box optimization benchmarking BBOB-2009 , 2010, GECCO '10.

[7]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Tutorial , 2016, ArXiv.

[8]  Dirk V. Arnold,et al.  Active covariance matrix adaptation for the (1+1)-CMA-ES , 2010, GECCO.

[9]  J. Leo van Hemmen,et al.  Accelerating backpropagation through dynamic self-adaptation , 1996, Neural Networks.

[10]  Terry Jones,et al.  Fitness Distance Correlation as a Measure of Problem Difficulty for Genetic Algorithms , 1995, ICGA.

[11]  Nikolaus Hansen,et al.  Adaptive Encoding: How to Render Search Coordinate System Invariant , 2008, PPSN.

[12]  N. Hansen.  Invariance, Self-adaptation and Correlated Mutations in Evolution Strategies Invariance, Self-adaptation and Correlated Mutations in Evolution Strategies , 2000 .

[13]  Tom Schaul,et al.  Exponential natural evolution strategies , 2010, GECCO '10.

[14]  Dirk V. Arnold,et al.  Weighted multirecombination evolution strategies , 2006, Theor. Comput. Sci..

[15]  Nikolaus Hansen,et al.  Step-Size Adaption Based on Non-Local Use of Selection Information , 1994, PPSN.

[16]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[17]  Isao Ono,et al.  Bidirectional Relation between CMA Evolution Strategies and Natural Evolution Strategies , 2010, PPSN.

[18]  Olivier Teytaud,et al.  General Lower Bounds for Evolutionary Algorithms , 2006, PPSN.

[19]  Anne Auger,et al.  Mirrored Sampling and Sequential Selection for Evolution Strategies , 2010, PPSN.

[20]  Jens Jägersküpper.  Lower Bounds for Hit-and-Run Direct Search , 2007, SAGA.

[21]  Christian Igel,et al.  Efficient covariance matrix update for variable metric evolution strategies , 2009, Machine Learning.

[22]  Jens Jägersküpper,et al.  Design and Management of Complex Technical Processes and Systems by means of Computational Intelligence Methods General Lower Bounds for Randomized Direct Search with Isotropic Sampling , 2008 .

[23]  Nikolaus Hansen,et al.  Benchmarking a BI-population CMA-ES on the BBOB-2009 function testbed , 2009, GECCO '09.

[24]  Anne Auger,et al.  Reconsidering the progress rate theory for evolution strategies in finite dimensions , 2006, GECCO '06.

[25]  Hans-Georg Beyer,et al.  The Theory of Evolution Strategies , 2001, Natural Computing Series.

[26]  K. Steiglitz,et al.  Adaptive step size random search , 1968 .

[27]  Nikolaus Hansen,et al.  Evaluating the CMA Evolution Strategy on Multimodal Test Functions , 2004, PPSN.

[28]  Tom Schaul,et al.  Natural Evolution Strategies , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[29]  Nikolaus Hansen,et al.  Benchmarking a BI-population CMA-ES on the BBOB-2009 noisy testbed , 2009, GECCO '09.

[30]  Olivier Teytaud,et al.  Lower Bounds for Evolution Strategies Using VC-Dimension , 2008, PPSN.

[31]  Nikolaus Hansen,et al.  A restart CMA evolution strategy with increasing population size , 2005, 2005 IEEE Congress on Evolutionary Computation.

[32]  Dirk V. Arnold,et al.  Optimal Weighted Recombination , 2005, FOGA.

[33]  Nikolaus Hansen,et al.  An Analysis of Mutative -Self-Adaptation on Linear Fitness Functions , 2006, Evolutionary Computation.

[34]  Petros Koumoutsakos,et al.  A Method for Handling Uncertainty in Evolutionary Optimization With an Application to Feedback Control of Combustion , 2009, IEEE Transactions on Evolutionary Computation.

[35]  Raymond Ros,et al.  Benchmarking a weighted negative covariance matrix update on the BBOB-2010 noiseless testbed , 2010, GECCO '10.

[36]  Dirk V. Arnold,et al.  Improving Evolution Strategies through Active Covariance Matrix Adaptation , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[37]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[38]  Anne Auger,et al.  Log-Linear Convergence and Divergence of the Scale-Invariant (1+1)-ES in Noisy Environments , 2011, Algorithmica.

[39]  Nikolaus Hansen.  Adaptive Encoding for Optimization , 2008 .

[40]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .