A Survey of Monte Carlo Tree Search Methods

Monte Carlo tree search (MCTS) is a recently proposed search method that combines the precision of tree search with the generality of random sampling. It has received considerable interest due to its spectacular success in the difficult problem of computer Go, but has also proved beneficial in a range of other domains. This paper is a survey of the literature to date, intended to provide a snapshot of the state of the art after the first five years of MCTS research. We outline the core algorithm's derivation, impart some structure on the many variations and enhancements that have been proposed, and summarize the results from the key game and nongame domains to which MCTS methods have been applied. A number of open research questions indicate that the field is ripe for future work.

[1]  Jonathan Schaeffer,et al.  The History Heuristic and Alpha-Beta Search Enhancements in Practice , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Bruce Abramson,et al.  Expected-Outcome: A General Model of Static Evaluation , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Bernd Brügmann Max-Planck.  Monte Carlo Go , 1993 .

[4]  H. Jaap van den Herik,et al.  Proof-Number Search , 1994, Artif. Intell..

[5]  Nicolò Cesa-Bianchi,et al.  Gambling in a rigged casino: The adversarial multi-armed bandit problem , 1995, Proceedings of IEEE 36th Annual Foundations of Computer Science.

[6]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[7]  R. Agrawal.  Sample mean based index policies by O(log n) regret for the multi-armed bandit problem , 1995, Advances in Applied Probability.

[8]  S. LaValle.  Rapidly-exploring random trees : a new tool for path planning , 1998 .

[9]  Matthew L. Ginsberg,et al.  GIB: Imperfect Information in a Computationally Challenging Game , 2011, J. Artif. Intell. Res..

[10]  Brian Sheppard,et al.  World-championship-caliber Scrabble , 2002, Artif. Intell..

[11]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[12]  Jos W. H. M. Uiterwijk,et al.  Monte-Carlo tree search in production management problems , 2006 .

[13]  Tristan Cazenave,et al.  A Phantom-Go Program , 2006, ACG.

[14]  Sylvain Gelly,et al.  Exploration exploitation in Go: UCT for Monte-Carlo Go , 2006, NIPS 2006.

[15]  Olivier Teytaud,et al.  Modification of UCT with Patterns in Monte-Carlo Go , 2006 .

[16]  H. Jaap van den Herik,et al.  Monte-Carlo Proof-Number Search for Computer Go , 2006, Computers and Games.

[17]  Jan Willemson,et al.  Improved Monte-Carlo Search , 2006 .

[18]  Rémi Coulom,et al.  Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search , 2006, Computers and Games.

[19]  Bruno Bouzy,et al.  Move-Pruning Techniques for Monte-Carlo Go , 2006, ACG.

[20]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[21]  Guillaume Chaslot,et al.  A Comparison of Monte-Carlo Methods for Phantom Go , 2007 .

[22]  Rémi Coulom.  Monte-Carlo Tree Search in Crazy Stone , 2007 .

[23]  David Silver,et al.  Combining online and offline knowledge in UCT , 2007, ICML '07.

[24]  Hilmar Finnsson,et al.  CADIA-Player : a general game playing agent , 2007 .

[25]  Peter Lewis,et al.  MOVE ORDERING VS HEAVY PLAYOUTS : WHERE SHOULD HEURISTICS BE APPLIED IN MONTE CARLO GO ? , 2007 .

[26]  Rémi Munos,et al.  Bandit Algorithms for Tree Search , 2007, UAI.

[27]  Rémi Coulom,et al.  Computing "Elo Ratings" of Move Patterns in the Game of Go , 2007, J. Int. Comput. Games Assoc..

[28]  T. Cazenave.  Evolving Monte-Carlo Tree Search Algorithms , 2007 .

[29]  Philip Hingston,et al.  Experiments with Monte Carlo Othello , 2007, 2007 IEEE Congress on Evolutionary Computation.

[30]  Tristan Cazenave,et al.  Playing the Right Atari , 2007, J. Int. Comput. Games Assoc..

[31]  Julien Kloetzer,et al.  The Monte-Carlo Approach in Amazons , 2007 .

[32]  Peter Drake,et al.  Heuristics in Monte Carlo Go , 2007, IC-AI.

[33]  Chjan C. Lim,et al.  The Monte Carlo Approach , 2007 .

[34]  T. Cazenave,et al.  On the Parallelization of UCT , 2007 .

[35]  T. Cazenave.  Reflexive Monte-Carlo Search , 2007 .

[36]  Jos W. H. M. Uiterwijk,et al.  Monte-Carlo Tree Search in Backgammon , 2007 .

[37]  Sylvain Gelly,et al.  Modifications of UCT and sequence-like simulations for Monte-Carlo Go , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[38]  Pieter Spronck,et al.  Monte-Carlo Tree Search: A New Framework for Game AI , 2008, AIIDE.

[39]  Kazunori Yamaguchi,et al.  Evaluation of Monte Carlo tree search and the application to Go , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[40]  Yoshiyuki Kotani,et al.  Combining final score with winning percentage by sigmoid function in Monte-Carlo simulations , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[41]  Rémi Munos,et al.  Adaptive play in Texas Hold'em Poker , 2008, ECAI.

[42]  S. Gelly,et al.  Combining expert, offline, transient and online knowledge in Monte-Carlo exploration , 2008 .

[43]  Olivier Teytaud,et al.  Continuous Lunches Are Free Plus the Design of Optimal Optimization Algorithms , 2010, Algorithmica.

[44]  T. Raiko,et al.  Application of UCT Search to the Connection Games of Hex, Y, *Star, and Renkula! , 2008 .

[45]  Yi Wang,et al.  Game Player Strategy Pattern Recognition and How UCT Algorithms Apply Pre-knowledge of Player's Strategy to Improve Opponent AI , 2008, 2008 International Conference on Computational Intelligence for Modelling Control & Automation.

[46]  H. Jaap van den Herik,et al.  Parallel Monte-Carlo Tree Search , 2008, Computers and Games.

[47]  Richard S. Sutton,et al.  Sample-based learning and search with permanent and transient memories , 2008, ICML '08.

[48]  T. Cazenave,et al.  Monte-Carlo Tree Search for General Game Playing , 2008 .

[49]  Ingo Althofer.  On the Laziness of Monte-Carlo Game Tree Search in Non-tight Situations , 2008 .

[50]  Mark H. M. Winands,et al.  Monte-Carlo Tree Search Solver , 2008, Computers and Games.

[51]  Scott D. Goodwin,et al.  Knowledge Generation for Improving Simulations in UCT for General Game Playing , 2008, Australasian Conference on Artificial Intelligence.

[52]  Nicolas Jouandeau,et al.  A Parallel Monte-Carlo Tree Search Algorithm , 2008, Computers and Games.

[53]  Hiroyuki Iida,et al.  A comparative study of solvers in Amazons endgames , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[54]  H. Jaap van den Herik,et al.  Single-Player Monte-Carlo Tree Search , 2008, Computers and Games.

[55]  Yngvi Björnsson,et al.  Simulation-Based Approach to General Game Playing , 2008, AAAI.

[56]  Csaba Szepesvári,et al.  Online Optimization in X-Armed Bandits , 2008, NIPS.

[57]  David Silver,et al.  Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (2008) Achieving Master Level Play in 9 × 9 Computer Go , 2022 .

[58]  H. Jaap van den Herik,et al.  Progressive Strategies for Monte-Carlo Tree Search , 2008 .

[59]  H. Jaap van den Herik,et al.  Cross-Entropy for Monte-Carlo Tree Search , 2008, J. Int. Comput. Games Assoc..

[60]  Scott D. Goodwin,et al.  Learning and knowledge generation in General Games , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[61]  Richard J. Lorentz.  Amazons Discover Monte-Carlo , 2008, Computers and Games.

[62]  Yi Wang,et al.  To Create Adaptive Game Opponent by Using UCT , 2008, 2008 International Conference on Computational Intelligence for Modelling Control & Automation.

[63]  Maarten P. D. Schadd,et al.  Addressing NP-Complete Puzzles with Monte-Carlo Methods 1 , 2008 .

[64]  Levente Kocsis,et al.  Transpositions and move groups in Monte Carlo tree search , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[65]  Olivier Teytaud,et al.  On the Parallelization of Monte-Carlo planning , 2008, ICINCO 2008.

[66]  Tristan Cazenave.  Multi-player Go , 2008, Computers and Games.

[67]  Nathan R. Sturtevant,et al.  An Analysis of UCT in Multi-Player Games , 2008, J. Int. Comput. Games Assoc..

[68]  Yasuhiro Tajima,et al.  An Othello evaluation function based on Temporal Difference Learning using probability of winning , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[69]  Guy Van den Broeck,et al.  Monte-Carlo Tree Search in Poker Using Expected Reward Distributions , 2009, ACML.

[70]  Olivier Teytaud,et al.  Creating an Upper-Confidence-Tree Program for Havannah , 2009, ACG.

[71]  Yngvi Björnsson,et al.  CadiaPlayer: A Simulation-Based General Game Player , 2009, IEEE Transactions on Computational Intelligence and AI in Games.

[72]  Gerald Tesauro,et al.  Monte-Carlo simulation balancing , 2009, ICML '09.

[73]  Markus Püschel,et al.  Bandit-based optimization on graphs with application to library performance tuning , 2009, ICML '09.

[74]  Thierry Moudenc,et al.  Introduction of a new paraphrase generation tool based on Monte-Carlo sampling , 2009, ACL.

[75]  Alan Fern,et al.  UCT for Tactical Assault Planning in Real-Time Strategy Games , 2009, IJCAI.

[76]  Tomáš Kozelek,et al.  Methods of MCTS and the game Arimaa , 2009 .

[77]  Tristan Cazenave,et al.  Nested Monte-Carlo Search , 2009, IJCAI.

[78]  Xiao Liu,et al.  To Create Intelligent Adaptive Game Opponent by Using Monte-Carlo for Tree Search , 2009, 2009 Fifth International Conference on Natural Computation.

[79]  Y. Björnsson,et al.  Simulation Control in General Game Playing Agents , 2009 .

[80]  Dawei Du,et al.  Monte-Carlo Tree Search and Computer Go , 2009, Advances in Information and Intelligent Systems.

[81]  Tzung-Pei Hong,et al.  A novel ontology for computer go knowledge management , 2009, 2009 IEEE International Conference on Fuzzy Systems.

[82]  Zhiqing Liu,et al.  Backpropagation Modification in Monte-Carlo Game Tree Search , 2009, 2009 Third International Symposium on Intelligent Information Technology Application.

[83]  Jean-Yves Audibert,et al.  Minimax Policies for Adversarial and Stochastic Bandits. , 2009, COLT 2009.

[84]  Olivier Teytaud,et al.  Grid Coevolution for Adaptive Simulations: Application to the Building of Opening Books in the Game of Go , 2009, EvoWorkshops.

[85]  J. Schaeffer,et al.  Comparing UCT versus CFR in Simultaneous Games , 2009 .

[86]  Tzung-Pei Hong,et al.  The Computational Intelligence of MoGo Revealed in Taiwan's Computer Go Tournaments , 2009, IEEE Transactions on Computational Intelligence and AI in Games.

[87]  Martin Müller,et al.  A Study of UCT and Its Enhancements in an Artificial Game , 2009, ACG.

[88]  Alan Fern,et al.  Lower Bounding Klondike Solitaire with Monte-Carlo Planning , 2009, ICAPS.

[89]  Kevin Waugh,et al.  Monte Carlo Sampling for Regret Minimization in Extensive Games , 2009, NIPS.

[90]  Mark H. M. Winands,et al.  Evaluation Function Based Monte-Carlo LOA , 2009, ACG.

[91]  F. Schadd.  Monte-Carlo Search Techniques in the Modern Board Game Thurn and Taxis , 2009 .

[92]  Tristan Cazenave,et al.  Utilisation de la recherche arborescente Monte-Carlo au Hex , 2009, Rev. d'Intelligence Artif..

[93]  David P. Helmbold,et al.  All-Moves-As-First Heuristics in Monte-Carlo Go , 2009, IC-AI.

[94]  Xiao Liu,et al.  To Create Intelligent Adaptive Neuro-Controller of Game Opponent from UCT-Created Data , 2009, 2009 Sixth International Conference on Fuzzy Systems and Knowledge Discovery.

[95]  Nathan R Sturtevant,et al.  Improving State Evaluation, Inference, and Search in Trick-Based Card Games , 2009, IJCAI.

[96]  Nicolas Jouandeau,et al.  Parallel Nested Monte-Carlo search , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[97]  Peter Drake.  The Last-Good-Reply Policy for Monte-Carlo Go , 2009, J. Int. Comput. Games Assoc..

[98]  Olivier Teytaud,et al.  Meta Monte-Carlo Tree Search for Automatic Opening Book Generation , 2009 .

[99]  Jos W. H. M. Uiterwijk,et al.  Using Intelligent Search Techniques to Play the Game Khet , 2009 .

[100]  Peter I. Cowling,et al.  Monte Carlo search applied to card selection in Magic: The Gathering , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[101]  Simon M. Lucas,et al.  A simple tree search method for playing Ms. Pac-Man , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[102]  Pieter Spronck,et al.  Monte-Carlo Tree Search in Settlers of Catan , 2009, ACG.

[103]  Tristan Cazenave.  Monte-Carlo Kakuro , 2009, ACG.

[104]  Martin Müller,et al.  Monte-Carlo Exploration for Deterministic Planning , 2009, IJCAI.

[105]  Arpad Rimmel.  Improvements and Evaluation of the Monte Carlo Tree Search Algorithm , 2009 .

[106]  Hiroyuki Iida,et al.  Playing Amazons Endgames , 2009, J. Int. Comput. Games Assoc..

[107]  Flavien Balbo,et al.  Using a monte-carlo approach for bus regulation , 2009, 2009 12th International IEEE Conference on Intelligent Transportation Systems.

[108]  Michèle Sebag,et al.  Boosting Active Learning to Optimality: A Tractable Monte-Carlo, Billiard-Based Algorithm , 2009, ECML/PKDD.

[109]  Robert Briesemeister.  Analysis and Implementation of the Game OnTop , 2009 .

[110]  Gian Piero Favini,et al.  Monte Carlo Tree Search Techniques in the Game of Kriegspiel , 2009, IJCAI.

[111]  O. Teytaud,et al.  Upper Confidence Trees and Billiards for Optimal Active Learning , 2009 .

[112]  Martin Müller,et al.  A Lock-Free Multithreaded Monte-Carlo Tree Search Algorithm , 2009, ACG.

[113]  Flavien Balbo,et al.  Monte-Carlo Bus Regulation , 2009 .

[114]  Michèle Sebag,et al.  Optimal robust expensive optimization is tractable , 2009, GECCO.

[115]  David Silver,et al.  Reinforcement learning and simulation-based search in computer go , 2009 .

[116]  Olivier Teytaud,et al.  On the huge benefit of quasi-random mutations for multimodal optimization with application to grid-based tuning of neurocontrollers , 2009, ESANN.

[117]  Olivier Teytaud,et al.  Adding Expert Knowledge and Exploration in Monte-Carlo Tree Search , 2009, ACG.

[118]  Hideki Imai,et al.  A study on security evaluation methodology for image-based biometrics authentication systems , 2009, 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems.

[119]  Nathan R. Sturtevant,et al.  Understanding the Success of Perfect Information Monte Carlo Sampling in Game Tree Search , 2010, AAAI.

[120]  Shimpei Matsumoto,et al.  Evaluation of Simulation Strategy on Single-Player Monte-Carlo Tree Search and its Discussion for a Practical Scheduling Problem , 2010 .

[121]  Ingo Althöfer,et al.  Automatic Generation and Evaluation of Recombination Games , 2010, J. Int. Comput. Games Assoc..

[122]  David Tom,et al.  Investigating UCT and RAVE: steps towards a more robust method , 2010 .

[123]  Martin Müller,et al.  Computational Experiments with the RAVE Heuristic , 2010, Computers and Games.

[124]  Feng Xiao,et al.  Pruning in UCT Algorithm , 2010, 2010 International Conference on Technologies and Applications of Artificial Intelligence.

[125]  Jean Méhat,et al.  UCD: Upper Confidence Bound for Rooted Directed Acyclic Graphs , 2010, 2010 International Conference on Technologies and Applications of Artificial Intelligence.

[126]  Yngvi Björnsson,et al.  Learning Simulation Control in General Game-Playing Agents , 2010, AAAI.

[127]  Mark H. M. Winands,et al.  Enhancements for Multi-Player Monte-Carlo Tree Search , 2010, Computers and Games.

[128]  Paolo Ciancarini,et al.  Monte Carlo tree search in Kriegspiel , 2010, Artif. Intell..

[129]  Bart Selman,et al.  Understanding Sampling-based Adversarial Search Methods , 2010, UAI 2010.

[130]  Bart Selman,et al.  Understanding Sampling Style Adversarial Search Methods , 2010, UAI.

[131]  Kazunori Yamaguchi,et al.  Evaluation of Game Tree Search Methods by Game Records , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[132]  Tristan Cazenave,et al.  Nested Monte-Carlo Expression Discovery , 2010, ECAI.

[133]  Thomas Hérault,et al.  Scalability and Parallelization of Monte-Carlo Tree Search , 2010, Computers and Games.

[134]  Fabien Teytaud,et al.  Multiple Overlapping Tiles for Contextual Monte Carlo Tree Search , 2010, EvoApplications.

[135]  Daisuke Takahashi,et al.  A Shogi Program Based on Monte-Carlo Tree Search , 2010, J. Int. Comput. Games Assoc..

[136]  Kloetzer Julien,et al.  Experiments in Monte-Carlo Amazons (ゲーム情報学(GI) Vol.2010-GI-24) , 2010 .

[137]  T. Cazenave.  Monte-Carlo approximation of temperature , 2010 .

[138]  Richard J. Lorentz.  Improving Monte-Carlo Tree Search in Havannah , 2010, Computers and Games.

[139]  Tristan Cazenave,et al.  Score Bounded Monte-Carlo Tree Search , 2010, Computers and Games.

[140]  Olivier Teytaud,et al.  Parameter Tuning by Simple Regret Algorithms and Multiple Simultaneous Hypothesis Testing , 2010, ICINCO.

[141]  Ikuo Takeuchi,et al.  Parallel Monte-Carlo Tree Search with Simulation Servers , 2010, 2010 International Conference on Technologies and Applications of Artificial Intelligence.

[142]  Simon M. Lucas,et al.  A UCT agent for Tron: Initial investigations , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[143]  Olivier Teytaud,et al.  Consistency Modifications for Automatically Tuned Monte-Carlo Tree Search , 2010, LION.

[144]  Mark H. M. Winands,et al.  Monte Carlo Tree Search in Lines of Action , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[145]  Damien Pellier,et al.  An UCT Approach for Anytime Agent-Based Planning , 2010, PAAMS.

[146]  Hendrik Baier,et al.  The Power of Forgetting: Improving the Last-Good-Reply Policy in Monte Carlo Go , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[147]  Olivier Teytaud,et al.  On the huge benefit of decisive moves in Monte-Carlo Tree Search algorithms , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[148]  Andrew Crampton,et al.  Monte-Carlo Planning for Pathfinding in Real-Time Strategy Games , 2010 .

[149]  Martin Müller,et al.  Fuego—An Open-Source Framework for Board Games and Go Engine Based on Monte Carlo Tree Search , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[150]  Abdallah Saffidine.  Some Improvements for Monte-Carlo Tree Search, Game Description Language Compilati , 2010 .

[151]  Yngvi Björnsson,et al.  CadiaPlayer: Search-Control Techniques , 2011, KI - Künstliche Intelligenz.

[152]  Julien Kloetzer.  Monte-Carlo Opening Books for Amazons , 2010, Computers and Games.

[153]  Ryan B. Hayward,et al.  Monte Carlo Tree Search in Hex , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[154]  Marius Thomas Lindauer,et al.  Centurio, a General Game Player: Parallel, Java- and ASP-based , 2010, KI - Künstliche Intelligenz.

[155]  Chi Wan Sung,et al.  A Monte-Carlo approach for ghost avoidance in the Ms. Pac-Man game , 2010, 2010 2nd International IEEE Consumer Electronics Society's Games Innovations Conference.

[156]  Guillaume Chaslot,et al.  Integrating Opponent Models with Monte-Carlo Tree Search in Poker , 2010, Interactive Decision Theory and Game Theory.

[157]  Bart Selman,et al.  On Adversarial Search Spaces and Sampling-Based Planning , 2010, ICAPS.

[158]  Olivier Teytaud,et al.  Bandit-Based Genetic Programming , 2010, EuroGP.

[159]  Michèle Sebag,et al.  Feature Selection as a One-Player Game , 2010, ICML.

[160]  Jean Méhat,et al.  A Parallel General Game Player , 2010, KI - Künstliche Intelligenz.

[161]  Shih-Chieh Huang,et al.  Monte-Carlo Simulation Balancing in Practice , 2010, Computers and Games.

[162]  V. T. Rajan,et al.  Bayesian Inference in Monte-Carlo Tree Search , 2010, UAI.

[163]  Shih-Chieh Huang,et al.  Monte-Carlo Simulation Balancing Applied to 9x9 Go , 2010, J. Int. Comput. Games Assoc..

[164]  Stefan Edelkamp,et al.  Finding the Needle in the Haystack with Heuristically Guided Swarm Tree Search , 2010, MKWI.

[165]  H. Jaap van den Herik,et al.  The Drosophila Revisited , 2010, J. Int. Comput. Games Assoc..

[166]  Thomas J. Walsh,et al.  Integrating Sample-Based Planning and Model-Based Reinforcement Learning , 2010, AAAI.

[167]  Olivier Teytaud,et al.  Special Issue on Monte Carlo Techniques and Computer Go , 2010, IEEE Trans. Comput. Intell. AI Games.

[168]  Yuan Gao,et al.  Optimizing player's satisfaction through DDA of game AI by UCT for the Game Dead-End , 2010, 2010 Sixth International Conference on Natural Computation.

[169]  Shih-Chieh Huang,et al.  Time Management for Monte-Carlo Tree Search Applied to the Game of Go , 2010, 2010 International Conference on Technologies and Applications of Artificial Intelligence.

[170]  Jean Méhat,et al.  Combining UCT and Nested Monte Carlo Search for Single-Player General Game Playing , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[171]  Osamu Watanabe,et al.  Evaluating Root Parallelization in Go , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[172]  Richard B. Segal,et al.  On the Scalability of Parallel UCT , 2010, Computers and Games.

[173]  Olivier Teytaud,et al.  Biasing Monte-Carlo Simulations through RAVE Values , 2010, Computers and Games.

[174]  Guillaume Maurice Jean-Bernard Chaslot Chaslot,et al.  Monte-Carlo Tree Search , 2010 .

[175]  Stephen Roberts,et al.  Multi-Armed Bandit Bayesian Decision Making , 2010 .

[176]  Gareth M. J. Williams.  Determining Game Quality Through UCT Tree Shape Analysis , 2010 .

[177]  Tristan Cazenave,et al.  Monte-Carlo Hex , 2010 .

[178]  Joel Veness,et al.  Monte-Carlo Planning in Large POMDPs , 2010, NIPS.

[179]  Martin Müller.  Fuego-GB Prototype at the Human machine competition in Barcelona 2010: a Tournament Report and Analysis , 2010 .

[180]  Shang-Rong Tsai,et al.  Current Frontiers in Computer Go , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[181]  Olivier Teytaud,et al.  Intelligent Agents for the Game of Go , 2010, IEEE Computational Intelligence Magazine.

[182]  Kanako Komiya,et al.  Nested Monte-Carlo Search with AMAF Heuristic , 2010, 2010 International Conference on Technologies and Applications of Artificial Intelligence.

[183]  Joel Veness,et al.  A Monte-Carlo AIXI Approximation , 2009, J. Artif. Intell. Res..

[184]  Solomon Eyal Shimony,et al.  Repeated-Task Canadian Traveler Problem , 2011, SOCS.

[185]  D. Thuente,et al.  Tree Parallelization of Ary on a Cluster , 2011 .

[186]  Cameron Browne.  The Dangers of Random Playouts , 2011, J. Int. Comput. Games Assoc..

[187]  Chi Wan Sung,et al.  A Monte-Carlo approach for the endgame of Ms. Pac-Man , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[188]  Julian Togelius,et al.  Towards Procedural Strategy Game Generation: Evolving Complementary Unit Types , 2011, EvoApplications.

[189]  N. Sylvester,et al.  A Linear Classifier Outperforms UCT in 9 x 9 Go , 2011 .

[190]  Rémi Munos,et al.  Pure exploration in finitely-armed and continuous-armed bandits , 2011, Theor. Comput. Sci..

[191]  Mark H. M. Winands,et al.  αβ-based play-outs in Monte-Carlo Tree Search , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[192]  Michael L. Littman,et al.  Sample-Based Planning for Continuous Action Markov Decision Processes , 2011, ICAPS.

[193]  Ian D. Watson,et al.  Computer poker: A review , 2011, Artif. Intell..

[194]  Olivier Teytaud,et al.  Upper Confidence Trees with Short Term Partial Information , 2011, EvoApplications.

[195]  Alan Fern,et al.  Ensemble Monte-Carlo Planning: An Empirical Study , 2011, ICAPS.

[196]  Michael L. Littman,et al.  Learning is planning: near Bayes-optimal reinforcement learning via Monte-Carlo tree search , 2011, UAI.

[197]  Csaba Szepesvári,et al.  –armed Bandits , 2022 .

[198]  Olivier Teytaud,et al.  Lemmas on partial observation, with application to phantom games , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[199]  Marco Platzner,et al.  Parallel Monte-Carlo Tree Search for HPC Systems , 2011, Euro-Par.

[200]  Christopher D. Rosin,et al.  Nested Rollout Policy Adaptation for Monte Carlo Tree Search , 2011, IJCAI.

[201]  Peter I. Cowling,et al.  Determinization and information set Monte Carlo Tree Search for the card game Dou Di Zhu , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[202]  Olivier Teytaud,et al.  Revisiting Monte-Carlo Tree Search on a Normal Form Game: NoGo , 2011, EvoApplications.

[203]  Takeshi Ito,et al.  Monte-Carlo tree search in Ms. Pac-Man , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[204]  R. Ramanujan,et al.  On the Behavior of UCT in Synthetic Search Spaces , 2011 .

[205]  Nataliya Sokolovska,et al.  Continuous Upper Confidence Trees , 2011, LION.

[206]  Makoto Yokoo,et al.  Real-Time Solving of Quantified CSPs Based on Monte-Carlo Game Tree Search , 2011, IJCAI.

[207]  Olivier Teytaud,et al.  Computational and human intelligence in blind Go , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[208]  Bart Selman,et al.  Trade-Offs in Sampling-Based Adversarial Planning , 2011, ICAPS.

[209]  Mark H. M. Winands,et al.  Monte-Carlo Tree Search for the game of Scotland Yard , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[210]  Julian Togelius,et al.  Modelling and evaluation of complex scenarios with the Strategy Game Description Language , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[211]  M. Littman,et al.  Approaching Bayes-optimalilty using Monte-Carlo tree search , 2011 .

[212]  Kirk L. Kroeker.  A new benchmark for artificial intelligence , 2011, CACM.

[213]  Olivier Teytaud,et al.  Random positions in Go , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[214]  Rong Jin,et al.  Double Updating Online Learning , 2011, J. Mach. Learn. Res..

[215]  Bart Selman,et al.  Monte-Carlo Style UCT Search for Boolean Satisfiability , 2011, AI*IA.

[216]  David Auger,et al.  Multiple Tree for Partially Observable Monte-Carlo Tree Search , 2011, EvoApplications.

[217]  Fabien Teytaud,et al.  Optimization of the Nested Monte-Carlo Algorithm on the Traveling Salesman Problem with Time Windows , 2011, EvoApplications.

[218]  Cameron Browne.  Towards MCTS for Creative Domains , 2011, ICCC.

[219]  Martin Müller,et al.  A Local Monte Carlo Tree Search Approach in Deterministic Planning , 2011, AAAI.

[220]  David Silver,et al.  Monte-Carlo tree search and rapid action value estimation in computer Go , 2011, Artif. Intell..

[221]  Simon M. Lucas,et al.  Learning non-random moves for playing Othello: Improving Monte Carlo Tree Search , 2011, 2011 IEEE Conference on Computational Intelligence and Games (CIG'11).

[222]  Maarten P. D. Schadd.  Selective search in games of different complexity , 2011 .

[223]  Simon M. Lucas,et al.  Fast Approximate Max-n Monte Carlo Tree Search for Ms Pac-Man , 2011, IEEE Transactions on Computational Intelligence and AI in Games.

[224]  Leandro Soriano Marcolino,et al.  Multi-agent Monte Carlo Go , 2011, AAMAS.

[225]  M. Winands,et al.  Monte-Carlo Tree Search for the Simultaneous Move Game Tron , 2012 .

[226]  Richard J. Lorentz.  CASTRO Wins Havannah Tournament , 2012, J. Int. Comput. Games Assoc..

[227]  Ryan B. Hayward,et al.  MOHEX Wins Hex Tournament , 2012, J. Int. Comput. Games Assoc..

[228]  Ashish Sabharwal,et al.  Guiding Combinatorial Optimization with UCT , 2012, CPAIOR.

[229]  Tristan Cazenave,et al.  Monte-Carlo Expression Discovery , 2013, Int. J. Artif. Intell. Tools.

[230]  Shi-Jim Yen,et al.  GOLOIS Wins Phantom Go Tournament , 2013, J. Int. Comput. Games Assoc..

[231]  T. L. Lai Andherbertrobbins.  Asymptotically Efficient Adaptive Allocation Rules , 2022 .