Swarm Tetris: Applying particle swarm optimization to tetris

This paper investigates the applicability of swarm-based algorithms to the game of Tetris. This work proposes an approach to the problem in which neural network weight values are optimized using a particle swarm optimization (PSO) algorithm. Such an approach has not previously been demonstrated as feasible for Tetris. The reported experimental results show the learning progress of the algorithm, as well as a comparison against a hand-optimized Tetris playing algorithm. The results indicate that the Tetris agents show a continuous improvement over the course of training. Since the experimental focus was on the feasibility of the approach rather than optimizing performance, optimized PSO-based agents were found to be outperformed by the hand-optimized algorithm. However, the playing strategies of the two agents were compared and shown to be similar. The results indicate that a swarm-based approach is feasible, and warrants further investigation.

[1]  Andries Petrus Engelbrecht,et al.  Training Bao Game-Playing Agents using Coevolutionary Particle Swarm Optimization , 2006, 2006 IEEE Symposium on Computational Intelligence and Games.

[2]  Erik D. Demaine,et al.  Tetris is hard, even to approximate , 2002, Int. J. Comput. Geom. Appl..

[3]  Riccardo Poli,et al.  Evolutionary Solo Pong players , 2005, 2005 IEEE Congress on Evolutionary Computation.

[4]  Niko Bohm,et al.  An Evolutionary Approach to Tetris , 2005 .

[5]  Bruno Scherrer,et al.  Improvements on Learning Tetris with Cross Entropy , 2009, J. Int. Comput. Games Assoc..

[6]  A. Engelbrecht,et al.  A new locally convergent particle swarm optimiser , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[7]  Benjamin Van Roy,et al.  Tetris: A Study of Randomized Constraint Sampling , 2006 .

[8]  Nicholas Lundgaard,et al.  Reinforcement Learning and Neural Networks for Tetris , 2007 .

[9]  Heidi Burgiel,et al.  How to lose at Tetris , 1997, The Mathematical Gazette.

[10]  A.P. Engelbrecht,et al.  Learning to play games using a PSO-based competitive learning approach , 2004, IEEE Transactions on Evolutionary Computation.

[11]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[12]  Sham M. Kakade,et al.  A Natural Policy Gradient , 2001, NIPS.

[13]  Cornelis J. Franken,et al.  PSO-based coevolutionary Game Learning , 2004 .

[14]  Robert V. Hogg,et al.  Introduction to Mathematical Statistics. , 1966 .

[15]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[16]  Paulo Cortez,et al.  Particle swarms for feedforward neural network training , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[17]  John M Brzustowski,et al.  Can you win at TETRIS , 1992 .

[18]  Hajime Kita,et al.  State evaluation strategy for exemplar-based policy optimization of dynamic decision problems , 2007, 2007 IEEE Congress on Evolutionary Computation.

[19]  David B. Fogel,et al.  Co-evolving checkers-playing programs using only win, lose, or draw , 1999, Defense, Security, and Sensing.

[20]  Donald Carr,et al.  Adapting Reinforcement Learning to Tetris , 2005 .

[21]  Ashraf M. Abdelbar,et al.  Co-evolutionary particle swarm optimization applied to the 7/spl times/7 Seega game , 2004, 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No.04CH37541).

[22]  John N. Tsitsiklis,et al.  Feature-based methods for large scale dynamic programming , 2004, Machine Learning.

[23]  Andries Petrus Engelbrecht,et al.  Comparing PSO structures to learn the game of checkers from zero knowledge , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[24]  Landon Flom,et al.  Using a Genetic Algorithm to Weight an Evaluation Function for Tetris , 2005 .

[25]  Michail G. Lagoudakis,et al.  Least-Squares Methods in Reinforcement Learning for Control , 2002, SETN.

[26]  Peter J. Angeline,et al.  Genetically Optimizing The Speed of Programs Evolved to Play Tetris , 1996 .

[27]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[28]  David B. Fogel,et al.  Evolving an expert checkers playing program without using human expertise , 2001, IEEE Trans. Evol. Comput..

[29]  Donald E. Knuth,et al.  An Analysis of Alpha-Beta Pruning , 1975, Artif. Intell..

[30]  András Lörincz,et al.  Learning Tetris Using the Noisy Cross-Entropy Method , 2006, Neural Computation.

[31]  Jan Ramon,et al.  On the numeric stability of Gaussian processes regression for relational reinforcement learning , 2004, ICML 2004.

[32]  S.M. Lucas,et al.  Evolutionary computation and games , 2006, IEEE Computational Intelligence Magazine.

[33]  Andries Petrus Engelbrecht,et al.  Coevolving Probabilistic Game Playing Agents using Particle Swarm Optimization Algorithm , 2005, CIG.

[34]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[35]  Bruno Scherrer,et al.  Building Controllers for Tetris , 2009, J. Int. Comput. Games Assoc..

[36]  Roger Germundsson,et al.  A Tetris Controller : An Example of a Discrete Event Dynamic System , 1991 .

[37]  Riccardo Poli,et al.  Analysis of the publications on the applications of particle swarm optimisation , 2008 .

[38]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[39]  Amine M. Boumaza.  On the evolution of artificial Tetris players , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.