Recurrent Neural Networks: Some Systems-Theoretic Aspects

This paper provides an exposition of some recent research regarding system-theoretic aspects of continuous-time recurrent (dynamic) neural networks with sigmoidal activation functions. The class of systems is introduced and discussed, and a result is cited regarding their universal approximation properties. Known characterizations of controllability, observability, and parameter identifiability are reviewed, as well as a result on minimality. Facts regarding the computational power of recurrent nets are also mentioned. ∗Supported in part by US Air Force Grant AFOSR-94-0293

[1]  Héctor J. Sussmann,et al.  Uniqueness of the weights for minimal feedforward nets with a given input-output map , 1992, Neural Networks.

[2]  Hava T. Siegelmann,et al.  Analog computation via neural networks , 1993, [1993] The 2nd Israel Symposium on Theory and Computing Systems.

[3]  Allan Pinkus,et al.  Multilayer Feedforward Networks with a Non-Polynomial Activation Function Can Approximate Any Function , 1991, Neural Networks.

[4]  Eduardo D. Sontag,et al.  Neural Networks for Control , 1993 .

[5]  Eduardo D. Sontag,et al.  Complete controllability of continuous-time recurrent neural networks , 1997 .

[6]  P. D. Pra,et al.  Forward accessibility for recurrent neural networks , 1995, IEEE Trans. Autom. Control..

[7]  Eduardo D. Sontag,et al.  NEURAL NETS AS SYSTEMS MODELS AND CONTROLLERS , 1992 .

[8]  Hava T. Siegelmann,et al.  On the Computational Power of Neural Nets , 1995, J. Comput. Syst. Sci..

[9]  Rafal Zbikowski.  Lie Algebra of Recurrent Neural Networks and Identifiability , 1993, 1993 American Control Conference.

[10]  Eduardo Sontag,et al.  For neural networks, function determines form , 1992, [1992] Proceedings of the 31st IEEE Conference on Decision and Control.

[11]  Eduardo D. Sontag,et al.  State observability in recurrent neural networks , 1993, Proceedings of 32nd IEEE Conference on Decision and Control.

[12]  Eduardo D. Sontag,et al.  Mathematical Control Theory: Deterministic Finite Dimensional Systems , 1990 .