Evolving neural networks

Neuroevolution, i.e. evolution of artificial neural networks, has recently emerged as a powerful technique for solving challenging reinforcement learning problems. Compared to traditional (e.g. value-function based) methods, neuroevolution is especially strong in domains where the state of the world is not fully known: the state can be disambiguated through recurrency, and novel situations handled through pattern matching. In this tutorial, we will review (1) neuroevolution methods that evolve fixed-topology networks, network topologies, and network construction processes, (2) ways of combining traditional neural network learning algorithms with evolutionary methods, and (3) applications of neuroevolution to game playing, robot control, resource optimization, and cognitive science.

[1]  Stefano Nolfi,et al.  Good teaching inputs do not correspond to desired responses in ecological neural networks , 1994, Neural Processing Letters.

[2]  Risto Miikkulainen,et al.  Evolving Obstacle Avoidance Behavior in a Robot Arm , 1996 .

[3]  Shimon Whiteson,et al.  Comparing evolutionary and temporal difference methods in a reinforcement learning domain , 2006, GECCO.

[4]  Phil Husbands,et al.  Evolutionary robotics , 2014, Evolutionary Intelligence.

[5]  Risto Miikkulainen,et al.  Learning basic navigation for personal satellite assistant using neuroevolution , 2005, GECCO '05.

[6]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[7]  Charles Ofria,et al.  The sensitivity of HyperNEAT to different geometric representations of a problem , 2009, GECCO.

[8]  Shimon Whiteson,et al.  Evolutionary Function Approximation for Reinforcement Learning , 2006, J. Mach. Learn. Res..

[9]  Risto Miikkulainen,et al.  COOPERATIVE COEVOLUTION OF MULTI-AGENT SYSTEMS , 2001 .

[10]  E. Ruppin Evolutionary autonomous agents: A neuroscience perspective , 2002, Nature Reviews Neuroscience.

[11]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[12]  Marco Colombetti,et al.  Incremental Robot Shaping , 1998, Connect. Sci..

[13]  Brad Fullmer and Risto Miikkulainen Using Marker-Based Genetic Encoding Of Neural Networks To Evolve Finite-State Behaviour , 1991 .

[14]  Risto Miikkulainen,et al.  Adaptive Control Utilising Neural Swarming , 2002, GECCO.

[15]  Jeffrey L. Elman,et al.  Learning and Evolution in Neural Networks , 1994, Adapt. Behav..

[16]  Risto Miikkulainen,et al.  Discovering Complex Othello Strategies through Evolutionary Neural Networks , 1995, Connect. Sci..

[17]  Risto Miikkulainen,et al.  Evolving symmetric and modular neural networks for distributed control , 2009, GECCO.

[18]  X. Yao Evolving Artificial Neural Networks , 1999 .

[19]  David J. Chalmers,et al.  The Evolution of Learning: An Experiment in Genetic Connectionism , 1991 .

[20]  Simon M. Lucas,et al.  A comparison of matrix rewriting versus direct encoding for evolving neural networks , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[21]  Kenneth O. Stanley,et al.  A Case Study on the Critical Role of Geometric Regularity in Machine Learning , 2008, AAAI.

[22]  G. M. Werner Evolution of Communication in Artificial Organisms, Artifial Life II , 1991 .

[23]  Risto Miikkulainen,et al.  Utilizing Domain Knowledge in Neuroevolution , 2003, ICML.

[24]  Risto Miikkulainen,et al.  Acquiring Visibly Intelligent Behavior with Example-Guided Neuroevolution , 2007, AAAI.

[25]  Kenneth A. De Jong,et al.  Cooperative Coevolution: An Architecture for Evolving Coadapted Subcomponents , 2000, Evolutionary Computation.

[26]  Chad W. Seys,et al.  Evolving Walking : The Anatomy of an Evolutionary Search , 2004 .

[27]  Alexis P. Wieland,et al.  Evolving Controls for Unstable Systems , 1991 .

[28]  Christian Igel,et al.  Neuroevolution for reinforcement learning using evolution strategies , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[29]  L. Darrell Whitley,et al.  Adding Learning to the Cellular Development of Neural Networks: Evolution and the Baldwin Effect , 1993, Evolutionary Computation.

[30]  Risto Miikkulainen,et al.  Cultural enhancement of neuroevolution , 2002 .

[31]  Risto Miikkulainen,et al.  Evolving Keepaway Soccer Players through Task Decomposition , 2003, GECCO.

[32]  Risto Miikkulainen,et al.  Efficient evolution of neural networks through complexification , 2004 .

[33]  Risto Miikkulainen,et al.  Evolving neural networks for fractured domains , 2008, GECCO '08.

[34]  Risto Miikkulainen,et al.  Incremental Evolution of Complex General Behavior , 1997, Adapt. Behav..

[35]  Risto Miikkulainen,et al.  Evolving neural network ensembles for control problems , 2005, GECCO '05.

[36]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..

[37]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[38]  Chris Aldrich,et al.  Intelligent process control utilising symbiotic memetic neuro-evolution , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[39]  Jimmy Secretan,et al.  Picbreeder: evolving pictures collaboratively online , 2008, CHI.

[40]  Risto Miikkulainen,et al.  Coevolution of neural networks using a layered pareto archive , 2006, GECCO.

[41]  Risto Miikkulainen,et al.  Accelerated Neural Evolution through Cooperatively Coevolved Synapses , 2008, J. Mach. Learn. Res..

[42]  J. D. Schaffer,et al.  Combinations of genetic algorithms and neural networks: a survey of the state of the art , 1992, [Proceedings] COGANN-92: International Workshop on Combinations of Genetic Algorithms and Neural Networks.

[43]  Risto Miikkulainen,et al.  Real-time neuroevolution in the NERO video game , 2005, IEEE Transactions on Evolutionary Computation.

[44]  Kenneth O. Stanley,et al.  Generative encoding for multiagent learning , 2008, GECCO '08.

[45]  Eric Harley,et al.  Blondie24, Playing at the Edge of AI (Book Review) , 2002, IEEE Intell. Informatics Bull..

[46]  Magnus Thor Jonsson,et al.  Evolution and design of distributed learning rules , 2000, 2000 IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks. Proceedings of the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (Cat. No.00.

[47]  Risto Miikkulainen,et al.  Efficient credit assignment through evaluation function decomposition , 2005, GECCO '05.

[48]  Risto Miikkulainen,et al.  Evolving Reusable Neural Modules , 2004, GECCO.

[49]  Risto Miikkulainen,et al.  Competitive Coevolution through Evolutionary Complexification , 2011, J. Artif. Intell. Res..

[50]  Risto Miikkulainen,et al.  Evolving explicit opponent models in game playing , 2007, GECCO '07.

[51]  Risto Miikkulainen,et al.  Forming Neural Networks Through Efficient and Adaptive Coevolution , 1997, Evolutionary Computation.

[52]  Francesco Mondada,et al.  Evolutionary neurocontrollers for autonomous mobile robots , 1998, Neural Networks.

[53]  Risto Miikkulainen,et al.  Neuro-evolution and natural deduction , 2000, 2000 IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks. Proceedings of the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (Cat. No.00.

[54]  Kenneth O. Stanley,et al.  Scaffolding for Interactively Evolving Novel Drum Tracks for Existing Songs , 2008, EvoWorkshops.

[55]  Michael G. Dyer,et al.  Evolution of herding behavior in artificial animals , 1993 .

[56]  Masahiro Fujita,et al.  Evolution of Controllers from a High-Level Simulator to a High DOF Robot , 2000, ICES.

[57]  Julian Togelius,et al.  Evolving robust and specialized car racing skills , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[58]  Risto Miikkulainen,et al.  Evolving a real-world vehicle warning system , 2006, GECCO.

[59]  Jude W. Shavlik,et al.  Creating Advice-Taking Reinforcement Learners , 1998, Machine Learning.

[60]  David H. Sharp,et al.  Scaling, machine learning, and genetic neural nets , 1989 .

[61]  David B. Fogel,et al.  Further Evolution of a Self-Learning Chess Program , 2005, CIG.

[62]  Chern Han Yong and Kenneth O. Stanley and Risto Miikkulainen Incorporating Advice into Evolution of Neural Networks , 2005 .

[63]  Risto Miikkulainen,et al.  Evolving a Roving Eye for Go , 2004, GECCO.

[64]  Risto Miikkulainen,et al.  Evolving neural networks for strategic decision-making problems , 2009, Neural Networks.

[65]  Richard K. Belew,et al.  Evolution, Learning, and Culture: Computational Metaphors for Adaptive Algorithms , 1990, Complex Syst..

[66]  Risto Miikkulainen,et al.  Modular neuroevolution for multilegged locomotion , 2008, GECCO '08.

[67]  Kenneth O. Stanley,et al.  A novel generative encoding for exploiting neural network sensor and output geometry , 2007, GECCO '07.

[68]  Eytan Ruppin,et al.  Emergence of Memory-Driven Command Neurons in Evolved Artificial Agents , 2001, Neural Computation.

[69]  John J. Grefenstette,et al.  Evolutionary Algorithms for Reinforcement Learning , 1999, J. Artif. Intell. Res..

[70]  Geoffrey E. Hinton,et al.  How Learning Can Guide Evolution , 1996, Complex Syst..

[71]  Inman Harvey,et al.  Explorations in Evolutionary Robotics , 1993, Adapt. Behav..

[72]  Risto Miikkulainen,et al.  Numerical optimization with neuroevolution , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[73]  Risto Miikkulainen,et al.  Acquiring evolvability through adaptive representations , 2007, GECCO '07.

[74]  Risto Miikkulainen,et al.  Neuroevolution for adaptive teams , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[75]  Risto Miikkulainen,et al.  Robust non-linear control through neuroevolution , 2003 .

[76]  Risto Miikkulainen,et al.  Creating melodies with evolving recurrent neural networks , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[77]  J. Baldwin A New Factor in Evolution , 1896, The American Naturalist.

[78]  Shimon Whiteson,et al.  Stochastic Optimization for Collision Selection in High Energy Physics , 2006, AAAI.

[79]  Risto Miikkulainen,et al.  Active Guidance for a Finless Rocket Using Neuroevolution , 2003, GECCO.

[80]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[81]  L. D. Whitley,et al.  Genetic Reinforcement Learning for Neurocontrol Problems , 2004, Machine Learning.

[82]  Risto Miikkulainen,et al.  A neuro-evolution method for dynamic resource allocation on a chip multiprocessor , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[83]  Donald E. Waagen,et al.  Evolving recurrent perceptrons for time-series modeling , 1994, IEEE Trans. Neural Networks.

[84]  Pattie Maes,et al.  Evolving Obstacle Avoidance Behavior in a Robot Arm , 1996 .

[85]  Lawrence Davis,et al.  Training Feedforward Neural Networks Using Genetic Algorithms , 1989, IJCAI.

[86]  Risto Miikkulainen,et al.  Automatic feature selection in neuroevolution , 2005, GECCO '05.

[87]  David B. Fogel,et al.  Evolution, neural networks, games, and intelligence , 1999, Proc. IEEE.

[88]  Isaac Meilijson,et al.  Axiomatic Scalable Neurocontroller Analysis via the Shapley Value , 2006, Artificial Life.

[89]  Jordi Madrenas,et al.  Evolvable Systems: From Biology to Hardware , 1996, Lecture Notes in Computer Science.

[90]  David E. Moriarty,et al.  Symbiotic Evolution of Neural Networks in Sequential Decision Tasks , 1997 .

[91]  Risto Miikkulainen,et al.  A Taxonomy for Artificial Embryogeny , 2003, Artificial Life.