Real-Time Evolution of Neural Networks in the NERO Video Game

A major goal for AI is to allow users to interact with agents that learn in real time, making new kinds of interactive simulations, training applications, and digital entertainment possible. This paper describes such a learning technology, called real-time NeuroEvolution of Augmenting Topologies (rtNEAT), and describes how rtNEAT was used to build the NeuroEvolving Robotic Operatives (NERO) video game. This game represents a new genre of machine learning games where the player trains agents in real time to perform challenging tasks in a virtual environment. Providing laymen the capability to effectively train agents in real time with no prior knowledge of AI or machine learning has broad implications, both in promoting the field of AI and making its achievements accessible to the public at large.

[1]  H. Johnson,et al.  A comparison of 'traditional' and multimedia information systems development practices , 2003, Inf. Softw. Technol..

[2]  Andrew W. Appel,et al.  Rog-O-Matic : a belligerent expert system , 1983 .

[3]  A. P. Wieland,et al.  Evolving neural network controllers for unstable systems , 1991, IJCNN-91-Seattle International Joint Conference on Neural Networks.

[4]  Byoung-Tak Zhang,et al.  Evolving Optimal Neural Networks Using Genetic Algorithms with Occam's Razor , 1993, Complex Syst..

[5]  David B. Fogel,et al.  Evolving Neural Control Systems , 1995, IEEE Expert.

[6]  Bir Bhanu,et al.  Using a Genetic Algorithm , 1995 .

[7]  Larry D. Pyeatt,et al.  A comparison between cellular encoding and direct encoding for genetic neural networks , 1996 .

[8]  David W. Opitz,et al.  Connectionist Theory Refinement: Genetically Searching the Space of Network Topologies , 1997, J. Artif. Intell. Res..

[9]  B. Habibi,et al.  Pengi : An Implementation of A Theory of Activity , 1998 .

[10]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[11]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[12]  Risto Miikkulainen,et al.  Solving Non-Markovian Control Tasks with Neuro-Evolution , 1999, IJCAI.

[13]  John E. Laird,et al.  Human-Level AI's Killer Application: Interactive Computer Games , 2000, AI Mag..

[14]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[15]  Benjamin Geisler,et al.  An Empirical Study of Machine Learning Algorithms Applied to Modeling Player Behavior in a "First Person Shooter" Video Game , 2002 .

[16]  Risto Miikkulainen,et al.  Competitive Coevolution through Evolutionary Complexification , 2011, J. Artif. Intell. Res..

[17]  Jaime R. Robles,et al.  using a genetic algorithm , 2004 .

[18]  Sung-Bae Cho,et al.  Evolution of emergent behaviors for shooting game characters in Robocode , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[19]  Sushil J. Louis,et al.  Using a genetic algorithm to tune first-person shooter bots , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[20]  David B. Fogel,et al.  A platform for evolving characters in competitive games , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[21]  Risto Miikkulainen,et al.  Real-time neuroevolution in the NERO video game , 2005, IEEE Transactions on Evolutionary Computation.

[22]  Risto Miikkulainen,et al.  Neuroevolution of an automobile crash warning system , 2005, GECCO '05.

[23]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[24]  Kenneth O. Stanley and Bobby D. Bryant and Risto Miikkulainen,et al.  Evolving Neural Network Agents in the NERO Video Game , 2005 .

[25]  Nicholas J. Radcliffe,et al.  Genetic set recombination and its application to neural network topology optimisation , 1993, Neural Computing & Applications.

[26]  Changchun Liu,et al.  An empirical study of machine learning techniques for affect recognition in human–robot interaction , 2006, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.