A Preliminary Study on Designing Artiicial Neural Networks Using Co-evolution

The design of optimal artiicial neural networks (ANNs) is a key issue in the study of ANNs from the point of view of both theory and applications. There are strong biological and engineering evidences to support that the information processing capability of an ANN is determined by its architecture. However, no systematic method for designing ANNs exists although there are many attempts in attacking this problem. This paper adopts an evolutionary approach to ANN design. The indirect encoding scheme of ANN ar-chitectures is used. That is, a genetic algorithm is used to evolve a set of grammar rules which generate an ANN architecture. A novel method of co-evolving a set of rules is proposed in this paper. In our co-evolutionary system, each individual in a population represents a rule. The whole population is the complete set of grammar rules which are used to generate an architecture. Preliminary experiments have been carried out to evolve ANN architectures for the parity problem with various sizes.

[1]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[2]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[3]  Robert E. Smith,et al.  Is a Learning Classifier System a Type of Neural Network? , 1994, Evolutionary Computation.

[4]  David H. Sharp,et al.  Scaling, machine learning, and genetic neural nets , 1989 .

[5]  D. Signorini,et al.  Neural networks , 1995, The Lancet.

[6]  Hiroaki Kitano,et al.  Designing Neural Networks Using Genetic Algorithms with Graph Generation System , 1990, Complex Syst..

[7]  Michael C. Mozer,et al.  Skeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment , 1988, NIPS.

[8]  James L. McClelland,et al.  James L. McClelland, David Rumelhart and the PDP Research Group, Parallel distributed processing: explorations in the microstructure of cognition . Vol. 1. Foundations . Vol. 2. Psychological and biological models . Cambridge MA: M.I.T. Press, 1987. , 1989, Journal of Child Language.

[9]  Yann LeCun,et al.  Optimal Brain Damage , 1989, NIPS.

[10]  Richard K. Belew,et al.  Evolving networks: using the genetic algorithm with connectionist learning , 1990 .

[11]  Frédéric Gruau,et al.  Genetic synthesis of Boolean neural networks with a cell rewriting developmental process , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[12]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[13]  Christian Lebiere,et al.  The Cascade-Correlation Learning Architecture , 1989, NIPS.

[14]  Marcus Frean,et al.  The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks , 1990, Neural Computation.

[15]  Yamashita,et al.  Backpropagation algorithm which varies the number of hidden units , 1989 .