Characterizing performance guarantees for multiagent, real-time systems operating in noisy and uncertain environments

Autonomous robots offer the potential to conduct Counter-Weapons of Mass Destruction (C-WMD) missions in an efficient and robust manner. However, to leverage this potential, a mission designer needs to be able to determine how well a robot system will operate in the noisy and uncertain environments that a C-WMD mission may require. We are developing a software framework for verification of performance guarantees for C-WMD missions based on the MissionLab software system and a novel process algebra approach to representing robot programs and operating environments. In this paper, we report on our initial research for the Defense Threat Reduction Agency (DTRA) in understanding what is required from a performance guarantee to give a mission designer the information necessary to understand how well a robot program will perform in a specific environment. We link this to prior work on metrics for robot performance. Using a simple mission scenario, we explore the implications of uncertainty in the four components of the problem: the robot program, and the sensors, actuators and environment with which the program is executed.

[1]  Hadas Kress-Gazit,et al.  Automatic synthesis of robot controllers for tasks with locative prepositions , 2010, 2010 IEEE International Conference on Robotics and Automation.

[2]  Jonathan P. Bowen,et al.  High-Integrity System Specification and Design , 1999, Formal Approaches to Computing and Information Technology (FACIT).

[3]  Ronald C. Arkin,et al.  Evaluating the Usability of Robot Programming Toolsets , 1998, Int. J. Robotics Res..

[4]  Jonathan P. Bowen,et al.  High-Integrity System Specification and Design , 1999 .

[5]  Edmund M. Clarke,et al.  Model Checking , 1999, Handbook of Automated Reasoning.

[6]  Ronald C. Arkin,et al.  An Behavior-based Robotics , 1998 .

[7]  Damian M. Lyons,et al.  Representing and analyzing action plans as networks of concurrent processes , 1993, IEEE Trans. Robotics Autom..

[8]  J. A. Valencia,et al.  Evaluation of navigation of an autonomous mobile robot , 2007 .

[9]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[10]  Ronald C. Arkin,et al.  Usability evaluation of high-level user assistance for robot mission specification , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[11]  Daniele Nardi,et al.  Performance evaluation of pure-motion tasks for mobile robots with respect to world models , 2009, Auton. Robots.

[12]  Ronald C. Arkin,et al.  Line-of-sight constrained exploration for reactive multiagent robotic teams , 2002, 7th International Workshop on Advanced Motion Control. Proceedings (Cat. No.02TH8623).

[13]  Stephan Merz,et al.  Model Checking , 2000 .

[14]  Alastair F. Donaldson,et al.  Software Model Checking , 2014, Computing Handbook, 3rd ed..

[15]  Jana Kosecka A framework for modeling and verifying visually guided agents: design, analysis and experiments , 1996 .

[16]  Damian M. Lyons,et al.  Towards performance guarantees for emergent behavior , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[17]  P. Ramadge,et al.  Supervisory control of a class of discrete event processes , 1987 .

[18]  Wolfram Burgard,et al.  Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) , 2005 .

[19]  Lynne E. Parker,et al.  Robot Teams: From Diversity to Polymorphism , 2002 .

[20]  Adam Jacoff,et al.  Using competitions to advance the development of standard test methods for response robots , 2012, PerMIS.

[21]  Ronald C. Arkin,et al.  Multiagent Mission Specification and Execution , 1997, Auton. Robots.

[22]  Raj Madhavan,et al.  Benchmarking and standardization of intelligent robotic systems , 2009, 2009 International Conference on Advanced Robotics.

[23]  Madhavan Raj Tunstel Performance Evaluation and Benchmarking of Intelligent Systems , 2013 .