EvoMCTS: Enhancing MCTS-based players through genetic programming

We present EvoMCTS, a genetic programming method for enhancing level of play in games. Our work focuses on the zero-sum, deterministic, perfect-information board game of Reversi. Expanding on our previous work on evolving board-state evaluation functions for alpha-beta search algorithm variants, we now evolve evaluation functions that augment the MTCS algorithm. We use strongly typed genetic programming, explicitly defined introns, and a selective directional crossover method. Our system regularly evolves players that outperform MCTS players that use the same amount of search. Our results prove scalable and EvoMCTS players whose search is increased offline still outperform MCTS counterparts. To demonstrate the generality of our method we apply EvoMCTS successfully to the game of Dodgem.

[1]  Moshe Sipper,et al.  Evolving board-game players with genetic programming , 2011, GECCO.

[2]  E. Berlekamp,et al.  Winning Ways for Your Mathematical Plays , 1983 .

[3]  Yngvi Björnsson,et al.  Simulation-Based Approach to General Game Playing , 2008, AAAI.

[4]  Risto Miikkulainen,et al.  Discovering Complex Othello Strategies through Evolutionary Neural Networks , 1995, Connect. Sci..

[5]  Moshe Sipper,et al.  Evolution of an Efficient Search Algorithm for the Mate-In-N Problem in Chess , 2007, EuroGP.

[6]  Moshe Sipper,et al.  GP-Gammon: Genetically Programming Backgammon Players , 2005, Genetic Programming and Evolvable Machines.

[7]  George C. Williams,et al.  Adaptation and Natural Selection , 2018 .

[8]  Jonathan Schaeffer,et al.  Checkers Is Solved , 2007, Science.

[9]  R. Lewontin ‘The Selfish Gene’ , 1977, Nature.

[10]  Yngvi Björnsson,et al.  CadiaPlayer: A Simulation-Based General Game Player , 2009, IEEE Transactions on Computational Intelligence and AI in Games.

[11]  Moshe Sipper,et al.  Evolving players that use selective game-tree search with genetic programming , 2012, GECCO '12.

[12]  David J. Montana,et al.  Strongly Typed Genetic Programming , 1995, Evolutionary Computation.

[13]  Donald E. Eastlake,et al.  The Greenblatt chess program , 1967, AFIPS '67 (Fall).

[14]  T. Cazenave Evolving Monte-Carlo Tree Search Algorithms , 2007 .

[15]  Moshe Sipper,et al.  GP-EndChess: Using Genetic Programming to Evolve Chess Endgame Players , 2005, EuroGP.

[16]  Moshe Sipper,et al.  Evolving Lose-Checkers players using genetic programming , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[17]  Kai-Fu Lee,et al.  The Development of a World Class Othello Program , 1990, Artif. Intell..

[18]  Rémi Coulom,et al.  Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search , 2006, Computers and Games.

[19]  Paul S. Rosenbloom,et al.  A World-Championship-Level Othello Program , 1982, Artif. Intell..

[20]  Jonathan Schaeffer,et al.  CHINOOK: The World Man-Machine Checkers Champion , 1996, AI Mag..

[21]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[22]  H. Jaap van den Herik,et al.  Progressive Strategies for Monte-Carlo Tree Search , 2008 .

[23]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[24]  Simon M. Lucas,et al.  Coevolution versus self-play temporal difference learning for acquiring position evaluation in small-board go , 2005, IEEE Transactions on Evolutionary Computation.

[25]  Ryan B. Hayward,et al.  MOHEX Wins Hex Tournament , 2012, J. Int. Comput. Games Assoc..

[26]  Sean Luke,et al.  Code Growth Is Not Caused by Introns , 2000 .

[27]  Olivier Teytaud,et al.  Modification of UCT with Patterns in Monte-Carlo Go , 2006 .

[28]  Ryan B. Hayward,et al.  Monte Carlo Tree Search in Hex , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[29]  Michael Buro,et al.  Improving heuristic mini-max search by supervised learning , 2002, Artif. Intell..

[30]  Kenneth O. Stanley,et al.  Evolving neural networks for geometric game-tree pruning , 2011, GECCO '11.