Performance analysis of the simultaneous perturbation stochastic approximation algorithm on the noisy sphere model

To theoretically compare the behavior of different algorithms, compatible performance measures are necessary. Thus in the first part, an analysis approach, developed for evolution strategies, was applied to simultaneous perturbation stochastic approximation on the noisy sphere model. A considerable advantage of this approach is that convergence results for non-noisy and noisy optimization can be obtained simultaneously. Next to the convergence rates, optimal step sizes and convergence criteria for 3 different noise models were derived. These results were validated by simulation experiments. Afterward, the results were used for a comparison with evolution strategies on the sphere model in combination with the 3 noise models. It was shown that both strategies perform similarly, with a slight advantage for SPSA if optimal settings are used and the noise strength is not too large.

[1]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[2]  James C. Spall.  Feedback and Weighting Mechanisms for Improving Jacobian Estimates in the Adaptive Simultaneous Perturbation Algorithm , 2009, IEEE Trans. Autom. Control..

[3]  E. Chong,et al.  A deterministic analysis of stochastic approximation with randomized directions , 1998, IEEE Transactions on Automatic Control.

[4]  H. Beyer,et al.  Performance of the -ES on a Class of PDQFs , 2010 .

[5]  Hans-Georg Beyer,et al.  Theory of evolution strategies - a tutorial , 2001 .

[6]  Ingo Rechenberg,et al.  The Evolution Strategy. A Mathematical Model of Darwinian Evolution , 1984 .

[7]  Dirk V. Arnold,et al.  Step Length Adaptation on Ridge Functions , 2008, Evolutionary Computation.

[8]  J. Spall,et al.  Theoretical comparisons of evolutionary computation and other optimization approaches , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[9]  Nikolaus Hansen,et al.  Step-Size Adaption Based on Non-Local Use of Selection Information , 1994, PPSN.

[10]  Hans-Georg Beyer,et al.  Self-Adaptation in Evolutionary Algorithms , 2007, Parameter Setting in Evolutionary Algorithms.

[11]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[12]  Irene A. Stegun,et al.  Pocketbook of mathematical functions , 1984 .

[13]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[14]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[15]  L. Gerencsér,et al.  SPSA in noise free optimization , 2000, Proceedings of the 2000 American Control Conference. ACC (IEEE Cat. No.00CH36334).

[16]  Nikolaus Hansen,et al.  The CMA Evolution Strategy: A Comparing Review , 2006, Towards a New Evolutionary Computation.

[17]  L. Gerencsér,et al.  The mathematics of noise-free SPSA , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[18]  James C. Spall,et al.  Adaptive stochastic approximation by the simultaneous perturbation method , 2000, IEEE Trans. Autom. Control..

[19]  Hans-Georg Beyer,et al.  Local performance of the (1 + 1)-ES in a noisy environment , 2002, IEEE Trans. Evol. Comput..

[20]  Hans-Paul Schwefel,et al.  Evolution strategies – A comprehensive introduction , 2002, Natural Computing.

[21]  Han-Fu Chen,et al.  A Kiefer-Wolfowitz algorithm with randomized differences , 1999, IEEE Trans. Autom. Control..

[22]  Hans-Georg Beyer,et al.  The Steady State Behavior of (µ/µI, lambda)-ES on Ellipsoidal Fitness Models Disturbed by Noise , 2003, GECCO.

[23]  Dirk V. Arnold,et al.  Optimal Weighted Recombination , 2005, FOGA.

[24]  C. T. Kelley,et al.  An Implicit Filtering Algorithm for Optimization of Functions with Many Local Minima , 1995, SIAM J. Optim..

[25]  J. Spall,et al.  Optimal random perturbations for stochastic approximation using a simultaneous perturbation gradient approximation , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[26]  Hans-Georg Beyer,et al.  A New Approach for Predicting the Final Outcome of Evolution Strategy Optimization Under Noise , 2005, Genetic Programming and Evolvable Machines.

[27]  Hans-Georg Beyer,et al.  On the behaviour of weighted multi-recombination evolution strategies optimising noisy cigar functions , 2009, GECCO '09.

[28]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[29]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[30]  Han-Fu Chen.  Stochastic approximation and its applications , 2002 .

[31]  Hans-Georg Beyer,et al.  The Theory of Evolution Strategies , 2001, Natural Computing Series.