Training multi-agent teams from zero knowledge with the competitive coevolutionary team-based particle swarm optimiser

A new competitive coevolutionary team-based particle swarm optimiser (CCPSO(t)) algorithm is developed to train multi-agent teams from zero knowledge. The CCPSO(t) algorithm is applied to train a team of agents to play simple soccer. The algorithm uses the charged particle swarm optimiser in a competitive and cooperative coevolutionary training environment to train neural network controllers for the players. The CCPSO(t) algorithm makes use of the FIFA league ranking relative fitness function to gather detailed performance metrics from each game played. The training performance and convergence behaviour of the particle swarm are analysed. A hypothesis is presented that explains the lack of convergence in the particle swarms. After applying a clustering algorithm on the particle positions, a detailed visual and quantitative analysis of the player strategies is presented. The final results show that the CCPSO(t) algorithm is capable of evolving complex gameplay strategies for a complex non-deterministic game.

[1]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[2]  Nelis Franken,et al.  Visual exploration of algorithm parameter space , 2009, 2009 IEEE Congress on Evolutionary Computation.

[3]  Peter J. Bentley,et al.  Don't push me! Collision-avoiding swarms , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[4]  Masoud Mohammadian New frontiers in computational intelligence and its applications , 2000 .

[5]  Larry H. Matthies,et al.  An autonomous path planner implemented on the Rocky 7 prototype microrover , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[6]  Tim M. Blackwell,et al.  Swarms in Dynamic Environments , 2003, GECCO.

[7]  Andries Petrus Engelbrecht,et al.  Evolving intelligent game-playing agents , 2004, South Afr. Comput. J..

[8]  Christiaan Scheepers,et al.  Coevolution of Neuro-controllers to Train Multi-Agent Teams from Zero Knowledge , 2013 .

[9]  Richard K. Belew,et al.  Methods for Competitive Co-Evolution: Finding Opponents Worth Beating , 1995, ICGA.

[10]  Andries Petrus Engelbrecht,et al.  Overfitting by PSO trained feedforward neural networks , 2010, IEEE Congress on Evolutionary Computation.

[11]  Andries Petrus Engelbrecht,et al.  Competitive coevolutionary training of simple soccer agents from zero knowledge , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[12]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[13]  Andries Petrus Engelbrecht,et al.  PSO approaches to coevolve IPD strategies , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[14]  Shengxiang Yang,et al.  Particle Swarm Optimization With Composite Particles in Dynamic Environments , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Andries Petrus Engelbrecht,et al.  Training Bao Game-Playing Agents using Coevolutionary Particle Swarm Optimization , 2006, 2006 IEEE Symposium on Computational Intelligence and Games.

[16]  Andries Petrus Engelbrecht,et al.  Comparing PSO structures to learn the game of checkers from zero knowledge , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[17]  A.P. Engelbrecht,et al.  Learning to play games using a PSO-based competitive learning approach , 2004, IEEE Transactions on Evolutionary Computation.

[18]  Hiroaki Kitano Massively Parallel Artificial Intelligence and Grand Challenge AI Applications. , 1994 .

[19]  Peter J. Bentley,et al.  Dynamic Search With Charged Swarms , 2002, GECCO.

[20]  Andrew W. Moore,et al.  X-means: Extending K-means with Efficient Estimation of the Number of Clusters , 2000, ICML.

[21]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[22]  T. Krink,et al.  Particle swarm optimisation with spatial particle extension , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).