Natural evolution strategies converge on sphere functions

This theoretical investigation gives the first proof of convergence for (radial) natural evolution strategies, on d-dimensional sphere functions, and establishes the conditions on hyper-parameters, as a function of d. For the limit case of large population sizes we show asymptotic linear convergence, and in the limit of small learning rates we give a full analytic characterization of the algorithm dynamics, decomposed into transient and asymptotic phases. Finally, we show why omitting the natural gradient is catastrophic.

[1]  M. Sankaran Approximations to the noncentral chi-square distribution , 1963 .

[2]  池田 信行,et al.  Stochastic differential equations and diffusion processes , 1981 .

[3]  L. Rogers Stochastic differential equations and diffusion processes: Nobuyuki Ikeda and Shinzo Watanabe North-Holland, Amsterdam, 1981, xiv + 464 pages, Dfl.175.00 , 1982 .

[4]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[5]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[6]  Anne Auger,et al.  Convergence results for the (1, lambda)-SA-ES using the theory of phi-irreducible Markov chains , 2005, Theor. Comput. Sci..

[7]  A. Auger Convergence results for the ( 1 , )-SA-ES using the theory of-irreducible Markov chains , 2005 .

[8]  Olivier Teytaud,et al.  General Lower Bounds for Evolutionary Algorithms , 2006, PPSN.

[9]  Tom Schaul,et al.  Natural Evolution Strategies , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[10]  Tom Schaul,et al.  Exponential natural evolution strategies , 2010, GECCO '10.

[11]  Isao Ono,et al.  Proposal of distance-weighted exponential natural evolution strategies , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[12]  Faustino J. Gomez,et al.  Novelty-based restarts for evolution strategies , 2011, 2011 IEEE Congress of Evolutionary Computation (CEC).

[13]  Tom Schaul,et al.  High dimensions and heavy tails for natural evolution strategies , 2011, GECCO '11.

[14]  Tom Schaul Studies in Continuous Black-box Optimization , 2011 .

[15]  Anne Auger,et al.  Theory of Evolution Strategies: A New Perspective , 2011, Theory of Randomized Search Heuristics.

[16]  Hans-Georg Beyer,et al.  Performance analysis of the simultaneous perturbation stochastic approximation algorithm on the noisy sphere model , 2012, Theor. Comput. Sci..