A Case Study on Improving Defense Behavior in Soccer Simulation 2D: The NeuroHassle Approach

While a lot of papers on RoboCup's robotic 2D soccer simulation have focused on the players' offensive behavior, there are only a few papers that specifically address a team's defense strategy. In this paper, we consider a defense scenario of crucial importance: We focus on situations where one of our players must interfere and disturb an opponent ball leading player in order to scotch the opponent team's attack at an early stage and, even better, to eventually conquer the ball initiating a counter attack. We employ a reinforcement learning methodology that enables our players to autonomously acquire such an aggressive duel behavior, and we have embedded it into our soccer simulation team's defensive strategy. Employing the learned NeuroHassle policy in our competition team, we were able to clearly improve the capabilities of our defense and, thus, to increase the performance of our team as a whole.

[1]  Martin A. Riedmiller,et al.  A direct adaptive method for faster backpropagation learning: the RPROP algorithm , 1993, IEEE International Conference on Neural Networks.

[2]  John N. Tsitsiklis,et al.  Neuro-Dynamic Programming , 1996, Encyclopedia of Machine Learning.

[3]  Hiroaki Kitano,et al.  RoboCup: A Challenge Problem for AI , 1997, AI Mag..

[4]  Ian Frank,et al.  Soccer Server: A Tool for Research on Multiagent Systems , 1998, Appl. Artif. Intell..

[5]  Luís Paulo Reis,et al.  Situation Based Strategic Positioning for Coordinating a Team of Homogeneous Agents , 2000, Balancing Reactivity and Social Deliberation in Multi-Agent Systems.

[6]  Enrico Pagello,et al.  Balancing Reactivity and Social Deliberation in Multi-Agent Systems , 2001, Lecture Notes in Computer Science.

[7]  David Steffen Bergman,et al.  Multi‐criteria optimization of ball passing in simulated soccer , 2005 .

[8]  Peter Stone,et al.  Reinforcement Learning for RoboCup Soccer Keepaway , 2005, Adapt. Behav..

[9]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[10]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[11]  Sahar Asadi,et al.  Dynamic Positioning Based on Voronoi Cells (DPVC) , 2005, RoboCup.

[12]  Adam Jacoff,et al.  RoboCup 2005: Robot Soccer World Cup IX , 2006, RoboCup.

[13]  Peter Stone,et al.  Half Field Offense in RoboCup Soccer: A Multiagent Reinforcement Learning Case Study , 2006, RoboCup.

[14]  Martin A. Riedmiller,et al.  Learning a Partial Behavior for a Competitive Robotic Soccer Agent , 2006, Künstliche Intell..

[15]  Vadim Kyrylov,et al.  While the ball in the digital soccer is rolling, where the non-player characters should go in a defensive situation? , 2007, Future Play.

[16]  Martin A. Riedmiller,et al.  On Experiences in a Complex and Competitive Gaming Domain: Reinforcement Learning Meets RoboCup , 2007, 2007 IEEE Symposium on Computational Intelligence and Games.

[17]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.