Adaptive model fitting with time-varying input variables

Consider the long-standing problem of fitting a model to multivariate data. In many control systems, we are interested in models that are associated with tracking a nonstationary process with time-varying input variables. It is often hopeless to produce a globally valid model over the whole domain in such a setting. Further, a globally valid model is not likely to be even needed in practice since some combinations of input variables are highly unlikely to occur. For this reason, we consider an adaptive model estimation method that emphasizes local fitting. This can be implemented in an elegant way using recursive methods such as stochastic approximation. The application motivating the general approach is the construction of real-time (or faster) training simulators for use by Navy personnel; the approach would apply in many other control and tracking applications.

[1]  A. O'Hagan,et al.  Curve Fitting and Optimal Design for Prediction , 1978 .

[2]  Jerome H. Friedman.  Multivariate adaptive regression splines (with discussion) , 1991 .

[3]  Eric Walter,et al.  Qualitative and quantitative experiment design for phenomenological models - A survey , 1990, Autom..

[4]  J. Spall,et al.  Optimal random perturbations for stochastic approximation using a simultaneous perturbation gradient approximation , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[5]  D. C. Chin.  The simultaneous perturbation method for processing magnetospheric images , 1998, Proceedings of the 1998 IEEE International Symposium on Intelligent Control (ISIC) held jointly with IEEE International Symposium on Computational Intelligence in Robotics and Automation (CIRA) Intell.

[6]  P. Chaudhuri,et al.  On efficient designing of nonlinear experiments , 1995 .

[7]  G. Pflug.  Stochastic minimization with constant step-size: asymptotic laws , 1986 .

[8]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[9]  S. D. Hill,et al.  Least-informative Bayesian prior distributions for finite samples based on information theory , 1990 .

[10]  J. Spall.  Stochastic Optimization, Stochastic Approximation and Simulated Annealing , 1999 .

[11]  J. Spall,et al.  Optimal random perturbations for stochastic approximation using a simultaneous perturbation gradient approximation , 1997, Proceedings of the 1997 American Control Conference (Cat. No.97CH36041).

[12]  D. C. Chin,et al.  Simultaneous perturbation method for processing magnetospheric images , 1999 .

[13]  M. A. Styblinski,et al.  Experiments in nonconvex optimization: Stochastic approximation with function smoothing and simulated annealing , 1990, Neural Networks.

[14]  J. Spall.  Implementation of the simultaneous perturbation algorithm for stochastic optimization , 1998 .

[15]  J. Friedman.  Multivariate adaptive regression splines , 1990 .

[16]  Arden Miller,et al.  Parameter Design for Signal-Response Systems: A Different Look at Taguchi's Dynamic Parameter Design , 1996 .

[17]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[18]  E. Eweda,et al.  Second-order convergence analysis of stochastic adaptive linear filtering , 1983 .