Stopping small-sample stochastic approximation

The practical application of stochastic approximation methods requires a reliable means to stop the iterative process when the estimate is close to the optimizer or when further improvement in the estimate is doubtful. Conventional ideas on stopping stochastic approximation algorithms employ criteria based on a proxy distribution — usually the asymptotic distribution. Yet difficulties may arise when applying such distributions to small (finite) samples. We propose an approach that uses the distribution of a statistically similar process called a surrogate for the proxy distribution rather than the asymptotic distribution. Under certain conditions, surrogate-based probability calculations are close to the actual probabilities. The question of how surrogate processes may be developed is also addressed. Two example applications are given.

[1]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[2]  James C. Spall,et al.  AN OVERVIEW OF THE SIMULTANEOUS PERTURBATION METHOD FOR EFFICIENT OPTIMIZATION , 1998 .

[3]  Mikhail Borisovich Nevelʹson,et al.  Stochastic Approximation and Recursive Estimation , 1976 .

[4]  S. D. Hill,et al.  Simulation optimization of airline delay with constraints , 2001, Proceeding of the 2001 Winter Simulation Conference (Cat. No.01CH37304).

[5]  Eugene Gilbo.  OPTIMIZATION OF AIR TRAFFIC MANAGEMENT STRATEGIES AT AIRPORTS WITH UNCERTAINTY IN AIRPORT CAPACITY , 1997 .

[6]  J. Spall.  A stochastic approximation algorithm for large-dimensional systems in the Kiefer-Wolfowitz setting , 1988, Proceedings of the 27th IEEE Conference on Decision and Control.

[7]  Stacy D. Hill,et al.  Airline and airport applications: simulation optimization of airline delay with constraints , 2001, WSC '01.

[8]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[9]  S. D. Hill,et al.  Simulation optimization of airline delay with constraints and multiple objectives , 2003, Fourth International Symposium on Uncertainty Modeling and Analysis, 2003. ISUMA 2003..

[10]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[11]  Abhijit Gosavi,et al.  Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning , 2003 .

[12]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[13]  James C. Spall,et al.  Stopping times and confidence bounds for small-sample stochastic approximation algorithms , 2009 .

[14]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[15]  H. Robbins,et al.  ON THE ASYMPTOTIC THEORY OF FIXED-WIDTH SEQUENTIAL CONFIDENCE INTERVALS FOR THE MEAN. , 1965 .

[16]  James C. Spall,et al.  Introduction to Stochastic Search and Optimization. Estimation, Simulation, and Control (Spall, J.C. , 2007 .

[17]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[18]  James C. Spall,et al.  Uncertainty Bounds in Parameter Estimation with Limited Data , 2005 .

[19]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[20]  Kenneth Geisinger.  AIRLINE DELAY, 1976-1986: BASED UPON THE STANDARDIZED DELAY REPORTING SYSTEM , 1989 .

[21]  Eugene P. Gilbo.  Optimizing airport capacity utilization in air traffic flow management subject to constraints at arrival and departure fixes , 1997, IEEE Trans. Control. Syst. Technol..