Implementation of the simultaneous perturbation algorithm for stochastic optimization

The need for solving multivariate optimization problems is pervasive in engineering and the physical and social sciences. The simultaneous perturbation stochastic approximation (SPSA) algorithm has recently attracted considerable attention for challenging optimization problems where it is difficult or impossible to directly obtain a gradient of the objective function with respect to the parameters being optimized. SPSA is based on an easily implemented and highly efficient gradient approximation that relies on measurements of the objective function, not on measurements of the gradient of the objective function. The gradient approximation is based on only two function measurements (regardless of the dimension of the gradient vector). This contrasts with standard finite-difference approaches, which require a number of function measurements proportional to the dimension of the gradient vector. This paper presents a simple step-by-step guide to implementation of SPSA in generic optimization problems and offers some practical suggestions for choosing certain algorithm coefficients.

[1]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[2]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[3]  F. Downton.  Stochastic Approximation , 1969, Nature.

[4]  M. T. Wasan.  Stochastic Approximation , 1969 .

[5]  J. Spall.  A Stochastic Approximation Technique for Generating Maximum Likelihood Parameter Estimates , 1987, 1987 American Control Conference.

[6]  R. Schnabel,et al.  A view of unconstrained optimization , 1989 .

[7]  H. White.  Some Asymptotic Results for Learning in Single Hidden-Layer Feedforward Network Models , 1989 .

[8]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[9]  James C. Spall,et al.  Developments in stochastic optimization algorithms with gradient approximations based on function measurements , 1994, Proceedings of Winter Simulation Conference.

[10]  Gert Cauwenberghs.  Analog VLSI autonomous systems for learning and optimization , 1994 .

[11]  D. C. Chin,et al.  A more efficient global optimization algorithm based on Styblinski and Tang , 1994, Neural Networks.

[12]  Michael C. Fu,et al.  Optimization via simulation: A review , 1994, Ann. Oper. Res..

[13]  J. Spall,et al.  Nonlinear adaptive control using neural networks: estimation with a smoothed form of simultaneous perturbation gradient approximation , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[14]  Fahimeh Rezayat.  On the use of an SPSA-based model-free controller in quality improvement , 1995, Autom..

[15]  Robert W. Brennan,et al.  Stochastic optimization applied to a manufacturing system operation problem , 1995, WSC '95.

[16]  Michael C. Fu,et al.  Transfer optimization via simultaneous perturbation stochastic approximation , 1995, WSC '95.

[17]  Yutaka Maeda,et al.  A learning rule of neural networks via simultaneous perturbation and its hardware implementation , 1995, Neural Networks.

[18]  M. Fu,et al.  Optimization of discrete event systems via simultaneous perturbation stochastic approximation , 1997 .

[19]  James C. Spall,et al.  A one-measurement form of simultaneous perturbation stochastic approximation , 1997, Autom..

[20]  J. Spall.  Accelerated second-order stochastic optimization using only function measurements , 1997, Proceedings of the 36th IEEE Conference on Decision and Control.

[21]  D. C. Chin,et al.  Comparative study of stochastic algorithms for system optimization based on gradient approximations , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[22]  Yangsheng Xu,et al.  Human control strategy: abstraction, verification, and replication , 1997 .

[23]  J. Dippon,et al.  Weighted Means in Stochastic Approximation of Minima , 1997 .

[24]  Thomas Parisini,et al.  Nonlinear modeling of complex large-scale plants using neural networks and stochastic approximation , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[25]  Rui J. P. de Figueiredo,et al.  Learning rules for neuro-controller via simultaneous perturbation , 1997, IEEE Trans. Neural Networks.

[26]  James C. Spall,et al.  A neural network controller for systems with unmodeled dynamics with applications to wastewater treatment , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[27]  Payman Sadegh,et al.  Constrained optimization via stochastic approximation with a simultaneous perturbation gradient approximation , 1997, Autom..

[28]  A Orman,et al.  Optimization of Stochastic Models: The Interface Between Simulation and Optimization , 2012, J. Oper. Res. Soc..

[29]  Hossein Arsham,et al.  Algorithms for sensitivity information in discrete-event systems simulation , 1998, Simul. Pract. Theory.

[30]  J. Spall,et al.  Optimal random perturbations for stochastic approximation using a simultaneous perturbation gradient approximation , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[31]  J. Spall,et al.  Model-free control of nonlinear stochastic systems with discrete-time measurements , 1998, IEEE Trans. Autom. Control..

[32]  J. C. Spall,et al.  Optimal sensor configuration for complex systems , 1998, Proceedings of the 1998 IEEE International Symposium on Intelligent Control (ISIC) held jointly with IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA) Intell.

[33]  J. Spall,et al.  Simulation-Based Optimization with Stochastic Approximation Using Common Random Numbers , 1999 .