Competitive coevolutionary training of simple soccer agents from zero knowledge

A new competitive coevolutionary team-based particle swarm optimisation (CCPSO) algorithm is developed to train multi-agent teams from zero knowledge. The CCPSO algorithm uses the charged particle swarm optimiser to train neural network controllers for simple soccer agents. The training performance of the CCPSO algorithm is analysed. The analysis identifies a critical weakness of the CCPSO algorithm in the form of outliers in the measured performance of the trained players. A hypothesis is presented that explains the presence of the outliers, followed by a detailed discussion of various biased and unbiased relative fitness functions. A new relative fitness function based on FIFA's league ranking system is presented. The performance of the unbiased relative fitness functions is evaluated and discussed. The final results show that the FIFA league ranking relative fitness function outperforms the other unbiased relative fitness functions, leading to consistent training results.

[1]  Wilfried Elmenreich,et al.  Evolving Neural Network Controllers for a Team of Self-Organizing Robots , 2010, J. Robotics.

[2]  Tim M. Blackwell,et al.  Swarms in Dynamic Environments , 2003, GECCO.

[3]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[4]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[5]  David B. Fogel,et al.  Blondie24: Playing at the Edge of AI , 2001 .

[6]  Andries Petrus Engelbrecht,et al.  PSO approaches to coevolve IPD strategies , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[7]  N. Franken,et al.  Evolving intelligent game-playing agents : research article , 2004 .

[8]  Peter J. Bentley,et al.  Don't push me! Collision-avoiding swarms , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[9]  Nelis Franken,et al.  Visual exploration of algorithm parameter space , 2009, 2009 IEEE Congress on Evolutionary Computation.

[10]  Ju-Jang Lee,et al.  Evolving multi-agents using a self-organizing genetic algorithm , 1997 .

[11]  Peter J. Bentley,et al.  Dynamic Search With Charged Swarms , 2002, GECCO.

[12]  Andries Petrus Engelbrecht,et al.  Comparing PSO structures to learn the game of checkers from zero knowledge , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[13]  Andries Petrus Engelbrecht,et al.  Training Bao Game-Playing Agents using Coevolutionary Particle Swarm Optimization , 2006, 2006 IEEE Symposium on Computational Intelligence and Games.

[14]  Richard K. Belew,et al.  Methods for Competitive Co-Evolution: Finding Opponents Worth Beating , 1995, ICGA.

[15]  Christiaan Scheepers,et al.  Coevolution of Neuro-controllers to Train Multi-Agent Teams from Zero Knowledge , 2013 .

[16]  David B. Fogel,et al.  Evolving neural networks to play checkers without relying on expert knowledge , 1999, IEEE Trans. Neural Networks.

[17]  A.P. Engelbrecht,et al.  Learning to play games using a PSO-based competitive learning approach , 2004, IEEE Transactions on Evolutionary Computation.

[18]  David B. Fogel,et al.  Anaconda defeats Hoyle 6-0: a case study competing an evolved checkers program against commercially available software , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).