Optimal random perturbations for stochastic approximation using a simultaneous perturbation gradient approximation

The simultaneous perturbation stochastic approximation (SPSA) algorithm has recently attracted considerable attention for optimization problems where it is difficult or impossible to obtain a direct gradient of the objective (say, loss) function. The approach is based on a highly efficient simultaneous perturbation approximation to the gradient based on loss function measurements. SPSA is based on picking a simultaneous perturbation (random) vector in a Monte Carlo fashion as part of generating the approximation to the gradient. This paper derives the optimal distribution for the Monte Carlo process. The objective is to minimize the mean square error of the estimate. We also consider maximization of the likelihood that the estimate be confined within a bounded symmetric region of the true parameter. The optimal distribution for the components of the simultaneous perturbation vector is found to be a symmetric Bernoulli in both cases. We end the paper with a numerical study related to the area of experiment design.

[1]  D. C. Chin,et al.  Comparative study of stochastic algorithms for system optimization based on gradient approximations , 1997, IEEE Trans. Syst. Man Cybern. Part B.

[2]  M. Fu,et al.  Transfer optimization via simultaneous perturbation stochastic approximation , 1995, Winter Simulation Conference Proceedings, 1995..

[3]  B. Palka An Introduction to Complex Function Theory , 1995 .

[4]  Thomas Parisini,et al.  Nonlinear modelling and state estimation in a real power plant using neural networks and stochastic approximation , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[5]  Fahimeh Rezayat On the use of an SPSA-based model-free controller in quality improvement , 1995, Autom..

[6]  Yutaka Maeda,et al.  A learning rule of neural networks via simultaneous perturbation and its hardware implementation , 1995, Neural Networks.

[7]  T. P. Krasulina,et al.  An algorithm for stochastic approximation with a preassigned probability of not exceeding a required threshold , 1995 .

[8]  Carsten SchererMechanical,et al.  Mixed H 2 =h 1 Control , 1995 .

[9]  Athanasios Sideris,et al.  H∞ optimization with time-domain constraints , 1994, IEEE Trans. Autom. Control..

[10]  M. Sznaier An exact solution to general SISO mixed H2/H∞ problems via convex optimization , 1994, IEEE Trans. Autom. Control..

[11]  Gert Cauwenberghs Analog VLSI autonomous systems for learning and optimization , 1994 .

[12]  D. C. Chin,et al.  A more efficient global optimization algorithm based on Styblinski and Tang , 1994, Neural Networks.

[13]  Cm Chien,et al.  AN ALTERNATIVE METHOD FOR DESIGN OF MIMO SYSTEM WITH LARGE PLANT UNCERTAINTY , 1993 .

[14]  Munther Dahleh,et al.  Solution to the general mixed H2/H∞ control problem - necessary conditions for optimality , 1992, 1992 American Control Conference.

[15]  J. Spall Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[16]  Pramod P. Khargonekar,et al.  H2-optimal control with an H∞-constraint The state feedback case , 1991, Autom..

[17]  Stephen P. Boyd,et al.  Linear controller design: limits of performance , 1991 .

[18]  P.-O. Gutman,et al.  An image processing approach for computing value sets of uncertain transfer functions , 1990, 29th IEEE Conference on Decision and Control.

[19]  Cm Chien,et al.  AN SISO UNCERTAIN SYSTEM DESIGNED BY AN EQUIVALENT DISTURBANCE ATTENUATION METHOD , 1990 .

[20]  Andrew C. Bartlett Nyquist, Bode, and Nichols Plots of Uncertain Systems , 1990, 1990 American Control Conference.

[21]  C.-H. Hui,et al.  A fast algorithm for computing parametric rational functions , 1989 .

[22]  P. Khargonekar,et al.  Approximation of infinite-dimensional systems , 1989 .

[23]  Guoxiang Gu,et al.  Two algorithms for frequency domain design of robust control systems , 1988 .

[24]  W. Haddad,et al.  LQG control with an H∞ performance bound: a riccati equation approach , 1988, 1988 American Control Conference.

[25]  J. Spall A Stochastic Approximation Technique for Generating Maximum Likelihood Parameter Estimates , 1987, 1987 American Control Conference.

[26]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[27]  V. Nollau Kushner, H. J./Clark, D. S., Stochastic Approximation Methods for Constrained and Unconstrained Systems. (Applied Mathematical Sciences 26). Berlin‐Heidelberg‐New York, Springer‐Verlag 1978. X, 261 S., 4 Abb., DM 26,40. US $ 13.20 , 1980 .

[28]  D.G. Dudley,et al.  Dynamic system identification experiment design and data analysis , 1979, Proceedings of the IEEE.

[29]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[30]  I. Horowitz,et al.  Synthesis of feedback systems with large plant ignorance for prescribed time-domain tolerances† , 1972 .

[31]  M. T. Wasan Stochastic Approximation , 1969 .

[32]  F. Downton Stochastic Approximation , 1969, Nature.