CadiaPlayer: A Simulation-Based General Game Player

The aim of general game playing (GGP) is to create intelligent agents that can automatically learn how to play many different games at an expert level without any human intervention. The traditional design model for GGP agents has been to use a minimax-based game-tree search augmented with an automatically learned heuristic evaluation function. The first successful GGP agents all followed that approach. In this paper, we describe CadiaPlayer, a GGP agent employing a radically different approach: instead of a traditional game-tree search, it uses Monte Carlo simulations for its move decisions. Furthermore, we empirically evaluate different simulation-based approaches on a wide variety of games, introduce a domain-independent enhancement for automatically learning search-control knowledge to guide the simulation playouts, and show how to adapt the simulation searches to be more effective in single-agent games. CadiaPlayer has already proven its effectiveness by winning the 2007 and 2008 Association for the Advancement of Artificial Intelligence (AAAI) GGP competitions.

[1]  Olivier Teytaud,et al.  Modification of UCT with Patterns in Monte-Carlo Go , 2006 .

[2]  H. Jaap van den Herik,et al.  Parallel Monte-Carlo Tree Search , 2008, Computers and Games.

[3]  Hilmar Finnsson,et al.  CADIA-Player : a general game playing agent , 2007 .

[4]  Peter Stone,et al.  Automatic Heuristic Construction in a Complete General Game Player , 2006, AAAI.

[5]  Tony Marsland,et al.  Selective depth-first game-tree search , 2002 .

[6]  T. Cazenave,et al.  On the Parallelization of UCT , 2007 .

[7]  Alexander Reinefeld,et al.  Enhanced Iterative-Deepening Search , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  David Silver,et al.  Combining online and offline knowledge in UCT , 2007, ICML '07.

[9]  James E. Clune,et al.  Heuristic Evaluation Functions for General Game Playing , 2007, KI - Künstliche Intelligenz.

[10]  Rémi Coulom,et al.  Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search , 2006, Computers and Games.

[11]  M. Buro,et al.  HOW MACHINES HAVE REARNEA TO PLAY OTHELLO , 1999 .

[12]  Murray Campbell,et al.  Deep Blue , 2002, Artif. Intell..

[13]  Jonathan Schaeffer,et al.  One jump ahead - challenging human supremacy in checkers , 1997, J. Int. Comput. Games Assoc..

[14]  Nicolas Jouandeau,et al.  A Parallel Monte-Carlo Tree Search Algorithm , 2008, Computers and Games.

[15]  M. R. Genesereth,et al.  Knowledge Interchange Format Version 3.0 Reference Manual , 1992, LICS 1992.

[16]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[17]  Stephan Schiffel,et al.  Automatic Construction of a Heuristic Search Function for General Game Playing , 2006 .

[18]  Bikramjit Banerjee and Gregory Kuhlmann and Peter Stone.  Value Function Transfer for General Game Playing , 2006 .

[19]  Yngvi Björnsson,et al.  Simulation-Based Approach to General Game Playing , 2008, AAAI.

[20]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[21]  Michael R. Genesereth,et al.  General Game Playing: Overview of the AAAI Competition , 2005, AI Mag..

[22]  Stephan Schiffel,et al.  Fluxplayer: A Successful General Game Player , 2007, AAAI.

[23]  Jonathan Schaeffer,et al.  The History Heuristic and Alpha-Beta Search Enhancements in Practice , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Nils J. Nilsson,et al.  A Formal Basis for the Heuristic Determination of Minimum Cost Paths , 1968, IEEE Trans. Syst. Sci. Cybern..

[25]  Bernhard Nebel,et al.  The FF Planning System: Fast Plan Generation Through Heuristic Search , 2011, J. Artif. Intell. Res..

[26]  Peter Stone,et al.  Graph-Based Domain Mapping for Transfer Learning in General Games , 2007, ECML.

[27]  Bikramjit Banerjee,et al.  General Game Learning Using Knowledge Transfer , 2007, IJCAI.

[28]  B. Pell.  A STRATEGIC METAGAME PLAYER FOR GENERAL CHESS‐LIKE GAMES , 1994, Comput. Intell..

[29]  Risto Miikkulainen,et al.  Coevolving Strategies for General Game Playing , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[30]  Kazunori Yamaguchi,et al.  Automatic Feature Construction and Optimization for General Game Player , 2001 .