An extension of the backpropagation method, termed dynamic backpropagation, which can be applied in a straightforward manner for the optimization of the weights (parameters) of multilayer neural networks is discussed. The method is based on the fact that gradient methods used in linear dynamical systems can be combined with backpropagation methods for neural networks to obtain the gradient of a performance index of nonlinear dynamical systems. The method can be applied to any complex system which can be expressed as the interconnection of linear dynamical systems and multilayer neural networks. To facilitate the practical implementation of the proposed method, emphasis is placed on the diagrammatic representation of the system which generates the gradient of the performance function.
[1]
K. Narendra,et al.
Multiparameter self-optimizing systems using correlation techniques
,
1964
.
[2]
J. Cruz,et al.
A new approach to the sensitivity problem in multivariable feedback system design
,
1964
.
[3]
Ronald J. Williams,et al.
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
,
1989,
Neural Computation.
[4]
Kumpati S. Narendra,et al.
Identification and control of dynamical systems using neural networks
,
1990,
IEEE Trans. Neural Networks.
[5]
Paul J. Werbos,et al.
Backpropagation Through Time: What It Does and How to Do It
,
1990,
Proc. IEEE.