Analysis of stagnation behaviour of competitive coevolutionary trained neuro-controllers

A new variant of the competitive coevolutionary team-based particle swarm optimiser (CCPSO(t)) algorithm is developed to train multi-agent teams from zero knowledge. Analysis show that the CCPSO algorithm stagnates during the training of simple soccer players. It is hypothesised that the stagnation is caused by saturation of the neural network weights. The CCPSO(t) algorithm is developed to overcome the stagnation problem. CCPSO(t) is based on the previously developed CCPSO algorithm with two additions. The first addition is the introduction of a restriction on the personal best particle positions. The second addition is the introduction of a linearly decreasing perception and core limit of the charged particle swarm optimiser. The final results show that the CCPSO(t) algorithm successfully addresses the CCPSO algorithm's neural network weight saturation problem.

[1]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[2]  Richard K. Belew,et al.  New Methods for Competitive Coevolution , 1997, Evolutionary Computation.

[3]  T. Krink,et al.  Particle swarm optimisation with spatial particle extension , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[4]  Peter J. Bentley,et al.  Dynamic Search With Charged Swarms , 2002, GECCO.

[5]  Andries Petrus Engelbrecht,et al.  Training Bao Game-Playing Agents using Coevolutionary Particle Swarm Optimization , 2006, 2006 IEEE Symposium on Computational Intelligence and Games.

[6]  Richard K. Belew,et al.  Methods for Competitive Co-Evolution: Finding Opponents Worth Beating , 1995, ICGA.

[7]  Andries Petrus Engelbrecht,et al.  Overfitting by PSO trained feedforward neural networks , 2010, IEEE Congress on Evolutionary Computation.

[8]  A.P. Engelbrecht,et al.  Learning to play games using a PSO-based competitive learning approach , 2004, IEEE Transactions on Evolutionary Computation.

[9]  Andries Petrus Engelbrecht,et al.  PSO approaches to coevolve IPD strategies , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[10]  Andries Petrus Engelbrecht,et al.  Comparing PSO structures to learn the game of checkers from zero knowledge , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[11]  Andries Petrus Engelbrecht,et al.  Competitive coevolutionary training of simple soccer agents from zero knowledge , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[12]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[13]  Peter J. Bentley,et al.  Don't push me! Collision-avoiding swarms , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[14]  Christiaan Scheepers,et al.  Coevolution of Neuro-controllers to Train Multi-Agent Teams from Zero Knowledge , 2013 .

[15]  Tim M. Blackwell,et al.  Swarms in Dynamic Environments , 2003, GECCO.

[16]  Andries Petrus Engelbrecht,et al.  Evolving intelligent game-playing agents , 2004, South Afr. Comput. J..