A linear time natural evolution strategy for non-separable functions

We present a novel Natural Evolution Strategy (NES) variant, the Rank-One NES (R1-NES), which uses a low-rank approximation of the search distribution covariance matrix. The algorithm allows computation of the natural gradient with cost linear in the dimensionality of the parameter space, and excels in solving high-dimensional non-separable problems.

[1]  Christian Igel,et al.  Registration of bone structures in 3D ultrasound and CT data: Comparison of different optimization strategies , 2005 .

[2]  Raymond Ros,et al.  A Simple Modification in CMA-ES Achieving Linear Time and Space Complexity , 2008, PPSN.

[3]  Anne Auger,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions , 2009 .

[4]  Bernhard Sendhoff,et al.  Three dimensional evolutionary aerodynamic design optimization with CMA-ES , 2005, GECCO '05.

[5]  Tom Schaul,et al.  High dimensions and heavy tails for natural evolution strategies , 2011, GECCO '11.

[6]  Tom Schaul,et al.  Exponential natural evolution strategies , 2010, GECCO '10.

[7]  Anne Auger,et al.  Identification of the isotherm function in chromatography using CMA-ES , 2007, 2007 IEEE Congress on Evolutionary Computation.

[8]  Hans-Paul Schwefel,et al.  TWO-PHASE NOZZLE AND HOLLOW CORE JET EXPERIMENTS. , 1970 .

[9]  Petros Koumoutsakos,et al.  A Method for Handling Uncertainty in Evolutionary Optimization With an Application to Feedback Control of Combustion , 2009, IEEE Transactions on Evolutionary Computation.

[10]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[11]  Isao Ono,et al.  Bidirectional Relation between CMA Evolution Strategies and Natural Evolution Strategies , 2010, PPSN.

[12]  Shun-ichi Amari,et al.  Natural Gradient Works Efficiently in Learning , 1998, Neural Computation.

[13]  Tom Schaul,et al.  Stochastic search using the natural gradient , 2009, ICML '09.

[14]  Ofer M. Shir,et al.  The second harmonic generation case-study as a gateway for es to quantum control problems , 2007, GECCO '07.

[15]  Raymond Ros,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Experimental Setup , 2009 .

[16]  Tom Schaul,et al.  Natural Evolution Strategies , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[17]  Kaare Brandt Petersen,et al.  The Matrix Cookbook , 2006 .