Numerical Methods for Stochastic Control Problems in Continuous Time

A powerful and usable class of methods for numerically approximating the solutions to optimal stochastic control problems for diffusion, reflected diffusion, or jump-diffusion models is discussed. The basic idea involves uconsistent approximation of the model by a Markov chain, and then solving an appropriate optimization problem for the Murkoy chain model. A general method for obtaining a useful approximation is given. All the standard classes of cost functions can be handled here, for illustrative purposes, discounted and average cost per unit time problems with both reflecting and nonreflecting diffusions are concentrated on. Both the drift and the variance can be controlled. Owing to its increasing importance and to lack of material on numerical methods, an application to the control of queueing and production systems in heavy traffic is developed in detail. The methods of proof of convergence are relatively simple, using only some basic ideas in the theory of weak convergence of a sequence of probabi...