A neural network controller for systems with unmodeled dynamics with applications to wastewater treatment

This paper considers the use of neural networks (NN's) in controlling a nonlinear, stochastic system with unknown process equations. The approach here is based on using the output error of the system to train the NN controller without the need to assume or construct a separate model (NN or other type) for the unknown process dynamics. To implement such a direct adaptive control approach, it is required that connection weights in the NN be estimated while the system is being controlled. As a result of the feedback of the unknown process dynamics, however, it is not possible to determine the gradient of the loss function for use in standard (backpropagation-type) weight estimation algorithms. In principle, stochastic approximation algorithms in the standard (Kiefer-Wolfowitz) finite-difference form can be used for this weight estimation since they are based on gradient approximations from available system output errors. However, these algorithms will generally require a prohibitive number of observed system outputs. Therefore, this paper considers the use of a new stochastic approximation algorithm for this weight estimation, which is based on a "simultaneous perturbation" gradient approximation. It is shown that this algorithm can greatly enhance the efficiency over more standard stochastic approximation algorithms based on finite-difference gradient approximations. The approach is illustrated on a simulated wastewater treatment system with stochastic effects and nonstationary dynamics.

[1]  Takayuki Yamada,et al.  Dynamic system identification using neural networks , 1993, IEEE Trans. Syst. Man Cybern..

[2]  J. Spall,et al.  Nonlinear adaptive control using neural networks: estimation with a smoothed form of simultaneous perturbation gradient approximation , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[3]  Ken-ichi Funahashi,et al.  On the approximate realization of continuous mappings by neural networks , 1989, Neural Networks.

[4]  K Y San,et al.  The design of controllers for batch bioreactors , 1988, Biotechnology and bioengineering.

[5]  Daniel Sbarbaro,et al.  Neural Networks for Nonlinear Internal Model Control , 1991 .

[6]  J. Spall.  Multivariate stochastic approximation using a simultaneous perturbation gradient approximation , 1992 .

[7]  Isabelle Guyon,et al.  Neural Network Implementation of Admission Control , 1990, NIPS.

[8]  H. Kushner,et al.  Asymptotic Properties of Stochastic Approximations with Constant Coefficients. , 1981 .

[9]  Denis Dochain,et al.  Adaptive identification and control algorithms for nonlinear bacterial growth systems , 1984, Autom..

[10]  S. Yakowitz.  A globally convergent stochastic approximation , 1993 .

[11]  V Van Breusegem,et al.  Implementation of an adaptive controller for the startup and steady‐state running of a biomethanation process operated in the CSTR mode , 1991, Biotechnology and bioengineering.

[12]  Model-free control of nonlinear stochastic systems in discrete time , 1995, Proceedings of 1995 34th IEEE Conference on Decision and Control.

[13]  D. S. Bayard.  A forward method for optimal stochastic nonlinear and adaptive control , 1991 .

[14]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[15]  T. Soderstrom,et al.  Stationary performance of linear stochastic systems under single step optimal control , 1982 .

[16]  Kurt Hornik,et al.  Convergence of learning algorithms with constant learning rates , 1991, IEEE Trans. Neural Networks.

[17]  Hideaki Sakai,et al.  A nonlinear regulator design in the presence of system uncertainties using multilayered neural network , 1991, IEEE Trans. Neural Networks.

[18]  K.M. Passino,et al.  Bridging the gap between conventional and intelligent control , 1993, IEEE Control Systems.

[19]  John F. Andrews,et al.  Dynamic models and control strategies for wastewater treatment processes , 1974 .

[20]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[21]  J Thibault,et al.  On‐line prediction of fermentation variables using neural networks , 1990, Biotechnology and bioengineering.

[22]  M. A. Styblinski,et al.  Experiments in nonconvex optimization: Stochastic approximation with function smoothing and simulated annealing , 1990, Neural Networks.

[23]  Kumpati S. Narendra,et al.  Gradient methods for the optimization of dynamical systems containing neural networks , 1991, IEEE Trans. Neural Networks.

[24]  D. C. Chin,et al.  A more efficient global optimization algorithm based on Styblinski and Tang , 1994, Neural Networks.

[25]  A. Benveniste,et al.  A measure of the tracking capability of recursive stochastic algorithms with constant gains , 1982 .

[26]  Stochastic Version Of Second-Order Optimization Using Only Function Measurements , 1996 .

[27]  J. Spall,et al.  Direct adaptive control of nonlinear systems using neural networks and stochastic approximation , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[28]  F.-C. Chen,et al.  Back-propagation neural networks for nonlinear self-tuning adaptive control , 1990, IEEE Control Systems Magazine.

[29]  J. Spall,et al.  Stochastic approximation for neural network weight estimation in the control of uncertain nonlinear systems , 1992, [Proceedings 1992] IJCNN International Joint Conference on Neural Networks.

[30]  Dejan J. Sobajic,et al.  Neural-net computing and the intelligent control of systems , 1992 .