Coevolution of neural networks using a layered pareto archive

The Layered Pareto Coevolution Archive (LAPCA) was recently proposed as an effective Coevolutionary Memory (CM) which, under certain assumptions, approximates monotonic progress in coevolution. In this paper, a technique is developed that interfaces the LAPCA algorithm with NeuroEvolution of Augmenting Topologies (NEAT), a method to evolve neural networks with demonstrated efficiency in game playing domains. In addition, the behavior of LAPCA is analyzed for the first time in a complex game-playing domain: evolving neural network controllers for the game Pong. The technique is shown to keep the total number of evaluations in the order of those required by NEAT, making it applicable to complex domains. Pong players evolved with a LAPCA and with the Hall of Fame (HOF) perform equally well, but the LAPCA is shown to require significantly less space than the HOF. Therefore, combining NEAT and LAPCA is found to be an effective approach to coevolution.

[1]  Paul R. Cohen,et al.  Empirical methods for artificial intelligence , 1995, IEEE Expert.

[2]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[3]  Dave Cliff,et al.  Tracking the Red Queen: Measurements of Adaptive Progress in Co-Evolutionary Simulations , 1995, ECAL.

[4]  Stefano Nolfi,et al.  God Save the Red Queen! Competition in Co-Evolutionary Robotics , 1997 .

[5]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[6]  John J. Grefenstette,et al.  Evolutionary Algorithms for Reinforcement Learning , 1999, J. Artif. Intell. Res..

[7]  Jordan B. Pollack,et al.  Symbiotic Combination as an Alternative to Sexual Recombination in Genetic Algorithms , 2000, PPSN.

[8]  Jordan B. Pollack,et al.  A Game-Theoretic Approach to the Simple Coevolutionary Algorithm , 2000, PPSN.

[9]  R. Watson,et al.  Pareto coevolution: using performance against coevolved opponents in a game as dimensions for Pareto selection , 2001 .

[10]  Jordan B. Pollack,et al.  A Mathematical Framework for the Study of Coevolution , 2002, FOGA.

[11]  Risto Miikkulainen,et al.  Efficient evolution of neural network topologies , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[12]  Risto Miikkulainen,et al.  Efficient Reinforcement Learning Through Evolving Neural Network Topologies , 2002, GECCO.

[13]  Risto Miikkulainen,et al.  Continual Coevolution Through Complexification , 2002, GECCO.

[14]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[15]  Edwin D. de Jong,et al.  Learning the Ideal Evaluation Function , 2003, GECCO.

[16]  Jordan B. Pollack,et al.  A Game-Theoretic Memory Mechanism for Coevolution , 2003, GECCO.

[17]  Peter Norvig,et al.  Artificial intelligence - a modern approach, 2nd Edition , 2003, Prentice Hall series in artificial intelligence.

[18]  Kenneth O. Stanley,et al.  Achieving High-Level Functionality through Complexification , 2003 .

[19]  Edwin D. de Jong,et al.  Towards a bounded Pareto-coevolution archive , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[20]  Edwin D. de Jong,et al.  Ideal Evaluation from Coevolution , 2004, Evolutionary Computation.

[21]  Risto Miikkulainen,et al.  Competitive Coevolution through Evolutionary Complexification , 2011, J. Artif. Intell. Res..

[22]  J. Pollack,et al.  Solution concepts in coevolutionary algorithms , 2004 .

[23]  Edwin D. de Jong,et al.  The Incremental Pareto-Coevolution Archive , 2004, GECCO.

[24]  Kenneth O. Stanley,et al.  Real-Time Evolution in the NERO Video Game (Winner of CIG 2005 Best Paper Award) , 2005, CIG.

[25]  Kenneth O. Stanley and Bobby D. Bryant and Risto Miikkulainen,et al.  Evolving Neural Network Agents in the NERO Video Game , 2005 .