Learning to play games using a PSO-based competitive learning approach

A new competitive approach is developed for learning agents to play two-agent games. This approach uses particle swarm optimizers (PSO) to train neural networks to predict the desirability of states in the leaf nodes of a game tree. The new approach is applied to the TicTacToe game, and compared with the performance of an evolutionary approach. A performance criterion is defined to quantify performance against that of players making random moves. The results show that the new PSO-based approach performs well as compared with the evolutionary approach.

[1]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[2]  David B. Fogel,et al.  Blondie24: Playing at the Edge of AI , 2001 .

[3]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[4]  Suganthan.  [IEEE 1999. Congress on Evolutionary Computation-CEC99 - Washington, DC, USA (6-9 July 1999)] Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406) - Particle swarm optimiser with neighbourhood operator , 1999 .

[5]  David B. Fogel,et al.  Evolving an expert checkers playing program without using human expertise , 2001, IEEE Trans. Evol. Comput..

[6]  Huihe Shao,et al.  An ANN's evolved by a new evolutionary system and its application , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[7]  A. Engelbrecht,et al.  A new locally convergent particle swarm optimiser , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[8]  R. Eberhart,et al.  Comparing inertia weights and constriction factors in particle swarm optimization , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[9]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[10]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[11]  Stuart J. Russell,et al.  Do the right thing , 1991 .

[12]  Russell C. Eberhart,et al.  Human tremor analysis using particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[13]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[14]  James Kennedy,et al.  Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[15]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[16]  Hans J. Berliner,et al.  Hitech Again Wins Pennsylvania Chess Championship , 1988, J. Int. Comput. Games Assoc..

[17]  Judea Pearl,et al.  The solution for the branching factor of the alpha-beta pruning algorithm and its optimality , 1982, CACM.

[18]  James F. Frenzel,et al.  Training product unit neural networks with genetic algorithms , 1993, IEEE Expert.

[19]  F. van den Bergh,et al.  Training product unit networks using cooperative particle swarm optimisers , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[20]  P. Suganthan.  Particle swarm optimiser with neighbourhood operator , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[21]  Andries Petrus Engelbrecht,et al.  Comparing PSO structures to learn the game of checkers from zero knowledge , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[22]  Murray Campbell,et al.  Singular Extensions: Adding Selectivity to Brute-Force Searching , 1990, Artif. Intell..

[23]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[24]  Andries Petrus Engelbrecht,et al.  Using neighbourhoods with the guaranteed convergence PSO , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[25]  Hans J. Berliner,et al.  The B* Tree Search Algorithm: A Best-First Proof Procedure , 1979, Artif. Intell..