Feedback and Weighting Mechanisms for Improving Jacobian Estimates in the Adaptive Simultaneous Perturbation Algorithm

It is known that a stochastic approximation (SA) analogue of the deterministic Newton-Raphson algorithm provides an asymptotically optimal or near-optimal form of stochastic search. However, directly determining the required Jacobian matrix (or Hessian matrix for optimization) has often been difficult or impossible in practice. This paper presents a general adaptive SA algorithm that is based on a simple method for estimating the Jacobian matrix while concurrently estimating the primary parameters of interest. Relative to prior methods for adaptively estimating the Jacobian matrix, the paper introduces two enhancements that generally improve the quality of the estimates for underlying Jacobian (Hessian) matrices, thereby improving the quality of the estimates for the primary parameters of interest. The first enhancement rests on a feedback process that uses previous Jacobian estimates to reduce the error in the current estimate. The second enhancement is based on an optimal weighting of per-iteration Jacobian estimates. From the use of simultaneous perturbations, the algorithm requires only a small number of loss function or gradient measurements per iteration - independent of the problem dimension - to adaptively estimate the Jacobian matrix and parameters of primary interest.

[1]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[2]  L. Gerencsér,et al.  The mathematics of noise-free SPSA , 2001, Proceedings of the 40th IEEE Conference on Decision and Control (Cat. No.01CH37228).

[3]  H. Teicher,et al.  Probability theory: Independence, interchangeability, martingales , 1978 .

[4]  J. Spall.  Adaptive stochastic approximation by the simultaneous perturbation method , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[5]  T. Apostol.  Mathematical Analysis , 1957 .

[6]  Shalabh Bhatnagar,et al.  Adaptive Newton-based multivariate smoothed functional algorithms for simulation optimization , 2007, TOMC.

[7]  E. Eweda,et al.  Second-order convergence analysis of stochastic adaptive linear filtering , 1983 .

[8]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[9]  G. Yin,et al.  Averaging procedures in adaptive filtering: an efficient approach , 1992 .

[10]  F. Downton.  Stochastic Approximation , 1969, Nature.

[11]  J. Spall,et al.  A modified second‐order SPSA optimization algorithm for finite samples , 2002 .

[12]  H. Kushner,et al.  Stochastic Approximation and Recursive Algorithms and Applications , 2003 .

[13]  D. Ruppert.  A Newton-Raphson Version of the Multivariate Robbins-Monro Procedure , 1985 .

[14]  Pierre L'Ecuyer,et al.  Global Stochastic Optimization with Low-Dispersion Point Sets , 1998, Oper. Res..

[15]  H. Piaggio.  Mathematical Analysis , 1955, Nature.

[16]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[17]  Shalabh Bhatnagar,et al.  Adaptive multivariate three-timescale stochastic approximation algorithms for simulation based optimization , 2005, TOMC.

[18]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[19]  J. L. Maryak,et al.  Global random optimization by simultaneous perturbation stochastic approximation , 2001, Proceedings of the 2001 American Control Conference. (Cat. No.01CH37148).

[20]  J. Spall.  Feedback and weighting mechanisms for improving Jacobian (Hessian) estimates in the adaptive simultaneous perturbation algorithm , 2006, 2006 American Control Conference.

[21]  C. Z. Wei.  Multivariate Adaptive Stochastic Approximation , 1987 .