From Control Model to Program: Investigating Robotic Aerial Vehicle Accidents with MAYDAY

With wide adoption of robotic aerial vehicles (RAVs), their accidents increasingly occur, calling for in-depth investigation of such accidents. Unfortunately, an inquiry to “why did my drone crash” often ends up with nowhere, if the root cause lies in the RAV’s control program, due to the key challenges in evidence and methodology: (1) Current RAVs’ flight log only records high-level vehicle control states and events, without recording control program execution; (2) The capability of “connecting the dots” – from controller anomaly to program variable corruption to program bug location – is lacking. To address these challenges, we develop MAYDAY, a crossdomain post-accident investigation framework by mapping control model to control program, enabling (1) in-flight logging of control program execution, and (2) traceback to the control-semantic bug that led to an accident, based on controland program-level logs. We have applied MAYDAY to ArduPilot, a popular open-source RAV control program that runs on a wide range of commodity RAVs. Our investigation of 10 RAV accidents caused by real ArduPilot bugs demonstrates that MAYDAY is able to pinpoint the root causes of these accidents within the program with high accuracy and minimum runtime and storage overhead. We also found 4 recently patched bugs still vulnerable and alerted the ArduPilot team.

[1]  D. Graham,et al.  The synthesis of "optimum" transient response: Criteria and standard forms , 1953, Transactions of the American Institute of Electrical Engineers, Part II: Applications and Industry.

[2]  David W. Binkley,et al.  Interprocedural slicing using dependence graphs , 1990, TOPL.

[3]  F. Pukelsheim.  The Three Sigma Rule , 1994 .

[4]  James R. Larus,et al.  Efficient path profiling , 1996, Proceedings of the 29th Annual IEEE/ACM International Symposium on Microarchitecture. MICRO 29.

[5]  P. N. Paraskevopoulos,et al.  Modern Control Engineering , 2001 .

[6]  Jeffrey K. Uhlmann,et al.  Unscented filtering and nonlinear estimation , 2004, Proceedings of the IEEE.

[7]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[8]  Adrian Perrig,et al.  SecVisor: a tiny hypervisor to provide lifetime kernel code integrity for commodity OSes , 2007, SOSP.

[9]  Saeid Habibi,et al.  The Smooth Variable Structure Filter , 2007, Proceedings of the IEEE.

[10]  Xuxian Jiang,et al.  Guest-Transparent Prevention of Kernel Rootkits with VMM-Based Memory Shadowing , 2008, RAID.

[11]  Lei Sun,et al.  A Framework for Self-Verification of Firmware Updates over the Air in Vehicle ECUs , 2008, 2008 IEEE Globecom Workshops.

[12]  Yuanyuan Zhou,et al.  PRES: probabilistic replay with execution sketching on multiprocessors , 2009, SOSP '09.

[13]  Trishul M. Chilimbi,et al.  HOLMES: Effective statistical debugging via efficient path profiling , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[14]  Srikanth V. Krishnamurthy,et al.  ARES: an anti-jamming reinforcement system for 802.11 networks , 2009, CoNEXT '09.

[15]  Ding Yuan,et al.  SherLog: error diagnosis by connecting clues from run-time logs , 2010, ASPLOS XV.

[16]  Patrick Th. Eugster,et al.  Efficient diagnostic tracing for wireless sensor networks , 2010, SenSys '10.

[17]  Adrian Perrig,et al.  VIPER: verifying the integrity of PERipherals' firmware , 2011, CCS '11.

[18]  Balazs Gati,et al.  Open source autopilot for academic research - The Paparazzi system , 2013, 2013 American Control Conference.

[19]  Xiangyu Zhang,et al.  High Accuracy Attack Provenance via Binary-based Execution Partition , 2013, NDSS.

[20]  Andrea C. Arpaci-Dusseau,et al.  Ffsck: the fast file system checker , 2013, FAST.

[21]  Emmett Witchel,et al.  InkTag: secure applications on an untrusted operating system , 2013, ASPLOS '13.

[22]  Xiangyu Zhang,et al.  LogGC: garbage collecting audit log , 2013, CCS.

[23]  Ben Liblit,et al.  Lightweight control-flow instrumentation and postmortem analysis in support of debugging , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[24]  Vijay Varadharajan,et al.  TrustLite: a security architecture for tiny embedded devices , 2014, EuroSys '14.

[25]  Gene Tsudik,et al.  A minimalist approach to Remote Attestation , 2014, 2014 Design, Automation & Test in Europe Conference & Exhibition (DATE).

[26]  Yongdae Kim,et al.  Rocking Drones with Intentional Sound Noise on Gyroscopic Sensors , 2015, USENIX Security Symposium.

[27]  Ahmad-Reza Sadeghi,et al.  HAFIX: Hardware-Assisted Flow Integrity eXtension , 2015, 2015 52nd ACM/EDAC/IEEE Design Automation Conference (DAC).

[28]  Grace Xingxin Gao,et al.  GPS Signal Authentication From Cooperative Peers , 2015, IEEE Transactions on Intelligent Transportation Systems.

[29]  George Candea,et al.  Failure sketching: a technique for automated root cause diagnosis of in-production failures , 2015, SOSP.

[30]  Will Dietz,et al.  Nested Kernel: An Operating System Architecture for Intra-Kernel Privilege Separation , 2015, ASPLOS.

[31]  Xiangyu Zhang,et al.  ProTracer: Towards Practical Provenance Tracing by Alternating Between Logging and Tainting , 2016, NDSS.

[32]  Yunheung Paek,et al.  HDFI: Hardware-Assisted Data-Flow Isolation , 2016, 2016 IEEE Symposium on Security and Privacy (SP).

[33]  Hao Wu,et al.  Controlling UAVs with Sensor Input Spoofing Attacks , 2016, WOOT.

[34]  Westley Weimer,et al.  An Uncrewed Aerial Vehicle Attack Scenario and Trustworthy Repair Architecture , 2016, 2016 46th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshop (DSN-W).

[35]  Jingling Xue,et al.  SVF: interprocedural static value-flow analysis in LLVM , 2016, CC.

[36]  Eric T. Matson,et al.  Drone forensic framework: Sensor and data identification and verification , 2017, 2017 IEEE Sensors Applications Symposium (SAS).

[37]  Peng Liu,et al.  Postmortem Program Analysis with Hardware-Enhanced Post-Crash Artifacts , 2017, USENIX Security Symposium.

[38]  Junfeng Yang,et al.  DeepXplore: Automated Whitebox Testing of Deep Learning Systems , 2017, SOSP.

[39]  Frank Breitinger,et al.  DROP (DRone Open source Parser) your drone: Forensic analysis of the DJI Phantom III , 2017, Digit. Investig..

[40]  Long Cheng,et al.  Orpheus: Enforcing Cyber-Physical Execution Semantics to Defend Against Data-Oriented Attacks , 2017, ACSAC.

[41]  Raja Sengupta,et al.  A power consumption model for multi-rotor small unmanned aircraft systems , 2017, 2017 International Conference on Unmanned Aircraft Systems (ICUAS).

[42]  David Kotz,et al.  Application Memory Isolation on Ultra-Low-Power MCUs , 2018, USENIX Annual Technical Conference.

[43]  Alagan Anpalagan,et al.  Anti-Jamming Communications Using Spectrum Waterfall: A Deep Reinforcement Learning Approach , 2017, IEEE Communications Letters.

[44]  Paul Grünbacher,et al.  Monitoring CPS at Runtime - A Case Study in the UAV Domain , 2018, 2018 44th Euromicro Conference on Software Engineering and Advanced Applications (SEAA).

[45]  Wen-Chuan Lee,et al.  Detecting Attacks Against Robotic Vehicles: A Control Invariant Approach , 2018, CCS.

[46]  Alessandro Orso,et al.  Enabling Refinable Cross-Host Attack Investigation with Efficient Data Flow Tagging and Tracking , 2018, USENIX Security Symposium.

[47]  Annibale Panichella,et al.  Testing Autonomous Cars for Feature Interaction Failures using Many-Objective Search , 2018, 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[48]  Xinyan Deng,et al.  Cross-Layer Retrofitting of UAVs Against Cyber-Physical Attacks , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[49]  Jens B. Schmitt,et al.  Crowd-GPS-Sec: Leveraging Crowdsourcing to Detect and Localize GPS Spoofing Attacks , 2018, 2018 IEEE Symposium on Security and Privacy (SP).

[50]  Afsoon Afzal,et al.  Crashing Simulated Planes is Cheap: Can Simulation Detect Robotics Bugs Early? , 2018, 2018 IEEE 11th International Conference on Software Testing, Verification and Validation (ICST).

[51]  Zhongshu Gu,et al.  Securing Real-Time Microcontroller Systems through Customized Memory View Switching , 2018, NDSS.

[52]  Qixin Wang,et al.  A System Identification Based Oracle for Control-CPS Software Fault Localization , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE).

[53]  Xinyan Deng,et al.  Sensitivity-based dynamic control frequency scheduling of quadcopter MAVs , 2019, Defense + Commercial Sensing.

[54]  Xinyan Deng,et al.  RVFuzzer: Finding Input Validation Bugs in Robotic Vehicles through Control-Guided Testing , 2019, USENIX Security Symposium.

[55]  Emmanuel Lemoine,et al.  Amazon Prime Air , 2019 .