Believable Bot Navigation via Playback of Human Traces

Imitation is a powerful and pervasive primitive underlying examples of intelligent behaviour in nature. Can we use it as a tool to help build artificial agents that behave like humans do? This question is studied in the context of the BotPrize competition, a Turing-like test where computer game bots compete by attempting to fool human judges into thinking they are just another human player. One problem faced by such bots is that of human-like navigation within the virtual world. This chapter describes the Human Trace Controller, a component of the \({UT{\char 94}2}\) bot which took second place in the BotPrize 2010 competition. The controller uses a database of recorded human games in order to quickly retrieve and play back relevant segments of human navigation behaviour. Empirical evidence suggests that the method of direct imitation allows the bot to effectively solve several navigation problems while moving in a human-like fashion.

[1]  G. L. Dirichlet Über die Reduction der positiven quadratischen Formen mit drei unbestimmten ganzen Zahlen. , 1850 .

[2]  Georges Voronoi Nouvelles applications des paramètres continus à la théorie des formes quadratiques. Premier mémoire. Sur quelques propriétés des formes quadratiques positives parfaites. , 1908 .

[3]  Stephen J. Garland,et al.  Algorithm 97: Shortest path , 1962, Commun. ACM.

[4]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[5]  Chris L. Jackins,et al.  Oct-trees and their use in representing three-dimensional objects , 1980 .

[6]  Stefan Schaal,et al.  Robot Learning From Demonstration , 1997, ICML.

[7]  G. Rizzolatti,et al.  Neurophysiological mechanisms underlying the understanding and imitation of action , 2001, Nature Reviews Neuroscience.

[8]  John E. Laird,et al.  Human-Level AI's Killer Application: Interactive Computer Games , 2000, AI Mag..

[9]  Joe Grant Bell,et al.  Unreal Tournament 2003 , 2002 .

[10]  H. Bekkering,et al.  Developmental psychology: Rational imitation in preverbal infants , 2002, Nature.

[11]  Manuela M. Veloso,et al.  GameBots: a flexible test bed for multiagent team research , 2002, CACM.

[12]  M. Matarić,et al.  Task Learning Through Imitation and Human-Robot Interaction , 2004 .

[13]  Christian Bauckhage,et al.  Synthesizing Movements for Computer Game Characters , 2004, DAGM-Symposium.

[14]  David W. Aha,et al.  Integrating Learning in Interactive Gaming Simulators , 2004 .

[15]  David W. Aha,et al.  TIELT: A Testbed for Gaming Environments , 2005, AAAI.

[16]  Monica N. Nicolescu,et al.  Task Learning through Imitation and Human-robot Interaction , 2005 .

[17]  Christopher G. Atkeson,et al.  Policies based on trajectory libraries , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[18]  Johannes Fürnkranz,et al.  Machine learning and games , 2006, Machine Learning.

[19]  Gita Reese Sukthankar,et al.  Simultaneous Team Assignment and Behavior Recognition from Spatio-Temporal Agent Traces , 2006, AAAI.

[20]  Christopher G. Atkeson,et al.  Transfer of policies based on trajectory libraries , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Vadim Bulitko,et al.  An evaluation of models for predicting opponent positions in first-person shooter video games , 2008, 2008 IEEE Symposium On Computational Intelligence and Games.

[22]  Daniele Loiacono,et al.  Learning drivers for TORCS through imitation using supervised methods , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[23]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[24]  José María Valls,et al.  Programming Robosoccer agents by modeling human behavior , 2009, Expert Syst. Appl..

[25]  Rudolf Kadlec,et al.  Pogamut 3 Can Assist Developers in Building AI (Not Only) for Their Videogame Agents , 2009, AGS.

[26]  Philip Hingston,et al.  A Turing Test for Computer Game Bots , 2009, IEEE Transactions on Computational Intelligence and AI in Games.

[27]  Jeff Orkin,et al.  Automatic learning and generation of social behavior from collective human gameplay , 2009, AAMAS.

[28]  Philip Hingston,et al.  A new design for a Turing Test for Bots , 2010, Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games.

[29]  H. Pao,et al.  Game Bot Detection via Avatar Trajectory Analysis , 2010, IEEE Transactions on Computational Intelligence and AI in Games.

[30]  R. Thawonmas,et al.  Outline of ICE-CEC2011 and Its Mechanism for Learning FPS Tactics , 2011 .

[31]  Risto Miikkulainen,et al.  Human-Like Combat Behaviour via Multiobjective Neuroevolution , 2012, Believable Bots.