Super mario evolution

We introduce a new reinforcement learning benchmark based on the classic platform game Super Mario Bros. The benchmark has a high-dimensional input space, and achieving a good score requires sophisticated and varied strategies. However, it has tunable difficulty, and at the lowest difficulty setting decent score can be achieved using rudimentary strategies and a small fraction of the input space. To investigate the properties of the benchmark, we evolve neural network-based controllers using different network architectures and input spaces. We show that it is relatively easy to learn basic strategies capable of clearing individual levels of low difficulty, but that these controllers have problems with generalization to unseen levels and with taking larger parts of the input space into account. A number of directions worth exploring for learning betterperforming strategies are discussed.

[1]  Jürgen Schmidhuber,et al.  LSTM recurrent networks learn simple context-free and context-sensitive languages , 2001, IEEE Trans. Neural Networks.

[2]  William V. Wright,et al.  A Theory of Fun for Game Design , 2004 .

[3]  Thore Graepel,et al.  LEARNING TO FIGHT , 2004 .

[4]  Julian Togelius,et al.  Evolving controllers for simulated car racing , 2005, 2005 IEEE Congress on Evolutionary Computation.

[5]  Simon M. Lucas,et al.  Evolving a Neural Network Location Evaluator to Play Ms. Pac-Man , 2005, CIG.

[6]  Julian Togelius,et al.  Evolving robust and specialized car racing skills , 2006, 2006 IEEE International Conference on Evolutionary Computation.

[7]  Julian Togelius,et al.  Towards automatic personalised content creation for racing games , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[8]  Gary B. Parker,et al.  The Evolution of Multi-Layer Neural Networks for the Control of Xpilot Agents , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[9]  Georgios N. Yannakakis,et al.  Real-time adaptation of augmented-reality games for optimizing player satisfaction , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[10]  Julian Togelius,et al.  An experiment in automatic game design , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[11]  Julian Togelius,et al.  Generating diverse opponents with multiobjective evolution , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[12]  Julian Togelius,et al.  The WCCI 2008 simulated car racing competition , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[13]  Rudolf Kadlec,et al.  Evoluce chovan´õ inteligentn´õch agentû u v poÿc´õtaÿcovych hrach Evolution of intelligent agent behaviour in computer games , 2008 .

[14]  Bobby D. Bryant,et al.  Visual control in quake II with a cyclic controller , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[15]  Tom Schaul,et al.  Scalable Neural Networks for Board Games , 2009, ICANN.

[16]  Kenneth O. Stanley,et al.  A Hypercube-Based Encoding for Evolving Large-Scale Neural Networks , 2009, Artificial Life.

[17]  Jan Koutník,et al.  NEAT in HyperNEAT Substituted with Genetic Programming , 2009, ICANNGA.

[18]  Julian Togelius,et al.  Ontogenetic and Phylogenetic Reinforcement Learning , 2009, Künstliche Intell..

[19]  Kenneth O. Stanley A Hypercube-Based Indirect Encoding for Evolving Large-Scale Neural Networks , 2009 .