Evolving Neural Networks through Augmenting Topologies

An important question in neuroevolution is how to gain an advantage from evolving neural network topologies along with weights. We present a method, NeuroEvolution of Augmenting Topologies (NEAT), which outperforms the best fixed-topology method on a challenging benchmark reinforcement learning task. We claim that the increased efficiency is due to (1) employing a principled method of crossover of different topologies, (2) protecting structural innovation using speciation, and (3) incrementally growing from minimal structure. We test this claim through a series of ablation studies that demonstrate that each component is necessary to the system as a whole and to each other. What results is signicantly faster learning. NEAT is also an important contribution to GAs because it shows how it is possible for evolution to both optimize and complexify solutions simultaneously, offering the possibility of evolving increasingly complex solutions over generations, and strengthening the analogy with biological evolution.

[1]  Lawrence Davis,et al.  Training Feedforward Neural Networks Using Genetic Algorithms , 1989, IJCAI.

[2]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[3]  C. L. Giles,et al.  Constructive learning of recurrent neural networks , 1993, IEEE International Conference on Neural Networks.

[4]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[5]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[6]  E. Spanier,et al.  Molecular biology of the gene(fourth edition). Vol. I. General principles. Menlo Park/California; Reading, Massachusetts; Don Mills, Ontario; Wokingham, U.K.; Amsterdam; Sydney; Singapore; Tokyo; Madrid; Bogota; Santiago; San Juan: The Benjamin/Cummings Publishing Company, Inc., 1987. 765 pp., DM 12 , 1988 .

[7]  Brad Fullmer and Risto Miikkulainen.  Using Marker-Based Genetic Encoding Of Neural Networks To Evolve Finite-State Behaviour , 1991 .

[8]  David B. Fogel,et al.  Evolving Neural Control Systems , 1995, IEEE Expert.

[9]  Rajendra Krishnan,et al.  2DELTA-GANN: A NEW APPROACH TO TRAINING NEURAL NETWORKS USING GENETIC ALGORITHMS , 1994 .

[10]  Byoung-Tak Zhang,et al.  Evolving Optimal Neural Networks Using Genetic Algorithms with Occam's Razor , 1993, Complex Syst..

[11]  Vittorio Maniezzo,et al.  Genetic evolution of the topology and weight distribution of neural networks , 1994, IEEE Trans. Neural Networks.

[12]  Frédéric Gruau,et al.  Genetic Synthesis of Modular Neural Networks , 1993, ICGA.

[13]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[14]  Mark D. Pendrith.  On Reinforcement Learning of Control Actions in Noisy and Non-Markovian Domains , 1994 .

[15]  L. Darrell Whitley,et al.  Genetic Reinforcement Learning for Neurocontrol Problems , 2004, Machine Learning.

[16]  John J. Grefenstette,et al.  A Coevolutionary Approach to Learning Sequential Decision Rules , 1995, ICGA.

[17]  A. P. Wieland,et al.  Evolving neural network controllers for unstable systems , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[18]  J. Urgen Branke.  Evolutionary Algorithms for Neural Network Design and Training , 1995 .

[19]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[20]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[21]  John Holland,et al.  Adaptation in Natural and Artificial Sys-tems: An Introductory Analysis with Applications to Biology , 1975 .

[22]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[23]  Xin Yao,et al.  Automatic modularization by speciation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[24]  Charles W. Anderson,et al.  Learning to Control an Inverted Pendulum with Connectionist Networks , 1988, 1988 American Control Conference.

[25]  David E. Moriarty,et al.  Symbiotic Evolution of Neural Networks in Sequential Decision Tasks , 1997 .

[26]  J E Darnell,et al.  Speculations on the early course of evolution. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[27]  Samir W. Mahfoud.  Niching methods for genetic algorithms , 1996 .

[28]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[29]  Andrew G. Barto,et al.  Reinforcement learning control , 1994, Current Opinion in Neurobiology.

[30]  Xin Yao,et al.  Towards designing artificial neural networks by evolution , 1998 .

[31]  D. R. McGregor,et al.  Designing application-specific neural networks using the structured genetic algorithm , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[32]  Risto Miikkulainen,et al.  Forming Neural Networks Through Efficient and Adaptive Coevolution , 1997, Evolutionary Computation.

[33]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[34]  Inman Harvey,et al.  The artificial evolution of adaptive behaviour , 1994 .

[35]  Xin Yao,et al.  A Preliminary Study on Designing Artiicial Neural Networks Using Co-evolution , 1995 .

[36]  Paul J. Darwen,et al.  Co-Evolutionary Learning by Automatic Modularisation with Speciation , 1996 .

[37]  C. Radding,et al.  Homologous pairing and strand exchange in genetic recombination. , 1982, Annual review of genetics.

[38]  Nolan H. Sigal,et al.  Genetic recombination: the nature of a crossed strand-exchange between two homologous DNA molecules. , 1972, Journal of molecular biology.

[39]  Risto Miikkulainen,et al.  Solving Non-Markovian Control Tasks with Neuro-Evolution , 1999, IJCAI.

[40]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning through Symbiotic Evolution , 1996, Machine Learning.

[41]  Risto Miikkulainen,et al.  Robust non-linear control through neuroevolution , 2003 .

[42]  David W. Opitz,et al.  Connectionist Theory Refinement: Genetically Searching the Space of Network Topologies , 1997, J. Artif. Intell. Res..

[43]  Larry D. Pyeatt,et al.  A comparison between cellular encoding and direct encoding for genetic neural networks , 1996 .

[44]  Dirk Thierens,et al.  Non-redundant genetic coding of neural networks , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[45]  Hussein A. Abbass,et al.  Speeding Up Backpropagation Using Multiobjective Evolutionary Algorithms , 2003, Neural Computation.

[46]  Riccardo Poli,et al.  Evolving the Topology and the Weights of Neural Networks Using a Dual Representation , 2004, Applied Intelligence.

[47]  Risto Miikkulainen,et al.  A Taxonomy for Artificial Embryogeny , 2003, Artificial Life.

[48]  C.W. Anderson,et al.  Learning to control an inverted pendulum using neural networks , 1989, IEEE Control Systems Magazine.

[49]  Jong-Hwan Kim,et al.  Evolutionary ordered neural network with a linked-list encoding scheme , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[50]  Martin Mandischer,et al.  Representation and Evolution of Neural Networks , 1993 .

[51]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.