AV-FUZZER: Finding Safety Violations in Autonomous Driving Systems

This paper proposes AV-FUZZER, a testing framework, to find the safety violations of an autonomous vehicle (AV) in the presence of an evolving traffic environment. We perturb the driving maneuvers of traffic participants to create situations in which an AV can run into safety violations. To optimally search for the perturbations to be introduced, we leverage domain knowledge of vehicle dynamics and genetic algorithm to minimize the safety potential of an AV over its projected trajectory. The values of the perturbation determined by this process provide parameters that define participants’ trajectories. To improve the efficiency of the search, we design a local fuzzer that increases the exploitation of local optima in the areas where highly likely safety-hazardous situations are observed. By repeating the optimization with significantly different starting points in the search space, AV-FUZZER determines several diverse AV safety violations. We demonstrate AV-FUZZER on an industrial-grade AV platform, Baidu Apollo, and find five distinct types of safety violations in a short period of time. In comparison, other existing techniques can find at most two. We analyze the safety violations found in Apollo and discuss their overarching causes.

[1]  Wei Zhan,et al.  A non-conservatively defensive strategy for urban autonomous driving , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[2]  Kyongsu Yi,et al.  Design and Evaluation of a Driving Mode Decision Algorithm for Automated Driving Vehicle on a Motorway , 2016 .

[3]  Lionel C. Briand,et al.  Testing advanced driver assistance systems using multi-objective search and neural networks , 2016, 2016 31st IEEE/ACM International Conference on Automated Software Engineering (ASE).

[4]  Ravishankar K. Iyer,et al.  ML-Based Fault Injection for Autonomous Vehicles: A Case for Bayesian Fault Injection , 2019, 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[5]  Guanpeng Li,et al.  Understanding Error Propagation in Deep Learning Neural Network (DNN) Accelerators and Applications , 2017, SC17: International Conference for High Performance Computing, Networking, Storage and Analysis.

[6]  David W. Coit,et al.  Multi-objective optimization using genetic algorithms: A tutorial , 2006, Reliab. Eng. Syst. Saf..

[7]  Xin Xu,et al.  Baidu Apollo Auto-Calibration System - An Industry-Level Data-Driven and Learning based Vehicle Longitude Dynamic Calibrating Algorithm , 2018, ArXiv.

[8]  Gregory Dudek,et al.  Generating Adversarial Driving Scenarios in High-Fidelity Simulators , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[9]  Xin He,et al.  Attacking Vision-based Perception in End-to-End Autonomous Driving Models , 2019, J. Syst. Archit..

[10]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[11]  Alexander Pretschner,et al.  Did We Test All Scenarios for Automated and Autonomous Driving Systems? , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[12]  Alberto L. Sangiovanni-Vincentelli,et al.  Scenic: a language for scenario specification and scene generation , 2018, PLDI.

[13]  Matei Ripeanu,et al.  Finding Resilience-Friendly Compiler Optimizations Using Meta-Heuristic Search Techniques , 2016, 2016 12th European Dependable Computing Conference (EDCC).

[14]  Franz Wotawa,et al.  Genetic Algorithm-Based Test Parameter Optimization for ADAS System Testing , 2019, 2019 IEEE 19th International Conference on Software Quality, Reliability and Security (QRS).

[15]  Katherine Rose Driggs-Campbell,et al.  Adaptive Stress Testing with Reward Augmentation for Autonomous Vehicle Validatio , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[16]  Kamyar Azizzadenesheli,et al.  Efficient Exploration Through Bayesian Deep Q-Networks , 2018, 2018 Information Theory and Applications Workshop (ITA).

[17]  Alexander Pretschner,et al.  Fitness Functions for Testing Automated and Autonomous Driving Systems , 2019, SAFECOMP.

[18]  Pradip Bose,et al.  Understanding Error Propagation in GPGPU Applications , 2016, SC16: International Conference for High Performance Computing, Networking, Storage and Analysis.

[19]  Nathan DeBardeleben,et al.  TensorFI: A Configurable Fault Injector for TensorFlow Applications , 2018, 2018 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW).

[20]  Mykel J. Kochenderfer,et al.  Adaptive Stress Testing for Autonomous Vehicles , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).

[21]  Michael A. Lones,et al.  Sean Luke: essentials of metaheuristics , 2011, Genetic Programming and Evolvable Machines.

[22]  Vrizlynn L. L. Thing,et al.  Autonomous Vehicle Security: A Taxonomy of Attacks and Defences , 2016, 2016 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData).

[23]  Oszkár Semeráth,et al.  Towards System-Level Testing with Coverage Guarantees for Autonomous Vehicles , 2019, 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS).

[24]  Jianfeng Gao,et al.  Efficient Exploration for Dialog Policy Learning with Deep BBQ Networks \& Replay Buffer Spiking , 2016, ArXiv.

[25]  Cristina Nita-Rotaru,et al.  Are Self-Driving Cars Secure? Evasion Attacks Against Deep Neural Networks for Steering Angle Prediction , 2019, 2019 IEEE Security and Privacy Workshops (SPW).

[26]  R. Haupt.  Optimum population size and mutation rate for a simple real genetic algorithm that optimizes array factors , 2000, IEEE Antennas and Propagation Society International Symposium. Transmitting Waves of Progress to the Next Millennium. 2000 Digest. Held in conjunction with: USNC/URSI National Radio Science Meeting (C.

[27]  Karthik Pattabiraman,et al.  Modeling Input-Dependent Error Propagation in Programs , 2018, 2018 48th Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[28]  Paolo Arcaini,et al.  Generating Avoidable Collision Scenarios for Testing Autonomous Driving Systems , 2020, 2020 IEEE 13th International Conference on Software Testing, Validation and Verification (ICST).

[29]  Lionel C. Briand,et al.  Testing Vision-Based Control Systems Using Learnable Evolutionary Algorithms , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[30]  Gordon Fraser,et al.  Automatically testing self-driving cars with search-based procedural content generation , 2019, ISSTA.

[31]  Junfeng Yang,et al.  DeepXplore: Automated Whitebox Testing of Deep Learning Systems , 2017, SOSP.

[32]  Ole J. Mengshoel,et al.  Adaptive Stress Testing: Finding Failure Events with Reinforcement Learning , 2018, ArXiv.

[33]  Guanpeng Li,et al.  BinFI: an efficient fault injector for safety-critical machine learning systems , 2019, SC.

[34]  Georgios Fainekos,et al.  Simulation-based Adversarial Test Generation for Autonomous Vehicles with Machine Learning Components , 2018, 2018 IEEE Intelligent Vehicles Symposium (IV).