Toward generating neural network structures for function approximation

Abstract This paper proposes an algorithm based on the back propagation procedure that dynamically configures the structure of feedforward multilayered neural networks and demonstrates its potential for control applications. The algorithm applies an intelligent generate-and-test procedure that evaluates the learning performance of the structure used and modify it accordingly by exploring different alternatives and selecting the most promising one. The algorithm modifies the structure of the neural networks by adding or deleting neurons/layers. The efficiency of the algorithm is demonstrated using several case studies with very promising results.

[1]  B. Widrow,et al.  Neural networks for self-learning control systems , 1990, IEEE Control Systems Magazine.

[2]  Jean-Pierre Nadal,et al.  Study of a Growth Algorithm for a Feedforward Network , 1989, Int. J. Neural Syst..

[3]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[4]  A. Sideris,et al.  A multilayered neural network controller , 1988, IEEE Control Systems Magazine.

[5]  Richard P. Lippmann,et al.  An introduction to computing with neural nets , 1987 .

[6]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[7]  Allon Guez,et al.  ART based adaptive pole placement for neurocontrollers , 1991, Neural Networks.

[8]  Ethem Alpaydin,et al.  GAL: Networks That Grow When They Learn and Shrink When They Forget , 1994, Int. J. Pattern Recognit. Artif. Intell..

[9]  Albert Y. Zomaya,et al.  Dynamic performance of robot manipulators under different operating conditions , 1990 .

[10]  Kumpati S. Narendra,et al.  Gradient methods for the optimization of dynamical systems containing neural networks , 1991, IEEE Trans. Neural Networks.

[11]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[12]  R. Paul.  Robot Manipulators: Mathematics, Programming, and Control , 1981 .

[13]  Jocelyn Sietsma,et al.  Creating artificial neural networks that generalize , 1991, Neural Networks.

[14]  Saul B. Gelfand,et al.  Analysis of gradient descent learning algorithms for multilayer feedforward neural networks , 1991 .

[15]  Timur Ash,et al.  Dynamic node creation in backpropagation networks , 1989 .

[16]  Joachim Diederich,et al.  Connectionist Recruitment Learning , 1988, ECAI.

[17]  Thomas P. Vogl,et al.  Rescaling of variables in back propagation learning , 1991, Neural Networks.

[18]  A. K. Mahalanabis,et al.  A Digital Algorithm for Near-Minimum-Time Control of Robot Manipulators , 1987 .

[19]  A. Bejczy.  Robot arm dynamics and control , 1974 .

[20]  Michael C. Mozer,et al.  Skeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment , 1988, NIPS.

[21]  Kumpati S. Narendra,et al.  Identification and control of dynamical systems using neural networks , 1990, IEEE Trans. Neural Networks.

[22]  Lorien Y. Pratt,et al.  Comparing Biases for Minimal Network Construction with Back-Propagation , 1988, NIPS.

[23]  Robert M. Pap,et al.  Handbook of neural computing applications , 1990 .

[24]  Ehud D. Karnin,et al.  A simple procedure for pruning back-propagation trained neural networks , 1990, IEEE Trans. Neural Networks.

[25]  Takayuki Yamada,et al.  Possibility of neural networks controller for robot manipulators , 1990, Proceedings., IEEE International Conference on Robotics and Automation.

[26]  Marcus Frean,et al.  The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks , 1990, Neural Computation.

[27]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[28]  David E. Rumelhart,et al.  Predicting the Future: a Connectionist Approach , 1990, Int. J. Neural Syst..

[29]  Esther Levin,et al.  Neural network architecture for adaptive system modeling and control , 1989, International 1989 Joint Conference on Neural Networks.

[30]  Yoshio Hirose,et al.  Backpropagation algorithm which varies the number of hidden units , 1991, International 1989 Joint Conference on Neural Networks.

[31]  Ethem Alpaydm.  Grow-and-Learn: An Incremental Method for Category Learning , 1990 .

[32]  David A. Moon,et al.  The Common List Object-Oriented Programming Language Standard , 1989, Object-Oriented Concepts, Databases, and Applications.

[33]  J. Nadal,et al.  Learning in feedforward layered networks: the tiling algorithm , 1989 .

[34]  A. I. Ethem Alpaydin.  Neural models of incremental supervised and unsupervised learning , 1990 .