VerifAI: A Toolkit for the Formal Design and Analysis of Artificial Intelligence-Based Systems

We present VerifAI, a software toolkit for the formal design and analysis of systems that include artificial intelligence (AI) and machine learning (ML) components. VerifAI particularly addresses challenges with applying formal methods to ML components such as perception systems based on deep neural networks, as well as systems containing them, and to model and analyze system behavior in the presence of environment uncertainty. We describe the initial version of VerifAI, which centers on simulation-based verification and synthesis, guided by formal models and specifications. We give examples of several use cases, including temporal-logic falsification, model-based systematic fuzz testing, parameter synthesis, counterexample analysis, and data set augmentation.

[1]  Swarat Chaudhuri,et al.  AI2: Safety and Robustness Certification of Neural Networks with Abstract Interpretation , 2018, 2018 IEEE Symposium on Security and Privacy (SP).

[2]  J. Halton.  On the efficiency of certain quasi-random sequences of points in evaluating multi-dimensional integrals , 1960 .

[3]  Sanjit A. Seshia,et al.  Compositional Falsification of Cyber-Physical Systems with Machine Learning Components , 2017, NFM.

[4]  Alberto L. Sangiovanni-Vincentelli,et al.  Scenic: a language for scenario specification and scene generation , 2018, PLDI.

[5]  Forrest N. Iandola,et al.  SqueezeDet: Unified, Small, Low Power Fully Convolutional Neural Networks for Real-Time Object Detection for Autonomous Driving , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Sanjit A. Seshia,et al.  VERIFAI: A Toolkit for the Design and Analysis of Artificial Intelligence-Based Systems , 2019, ArXiv.

[7]  Sriram Sankaranarayanan,et al.  S-TaLiRo: A Tool for Temporal Logic Falsification for Hybrid Systems , 2011, TACAS.

[8]  Wojciech Zaremba,et al.  OpenAI Gym , 2016, ArXiv.

[9]  Matthew Wicker,et al.  Feature-Guided Black-Box Safety Testing of Deep Neural Networks , 2017, TACAS.

[10]  Pravin Varaiya,et al.  Making intersections safer with I2V communication , 2018, Transportation Research Part C: Emerging Technologies.

[11]  Sanjit A. Seshia,et al.  Formal Specification for Deep Neural Networks , 2018, ATVA.

[12]  Sanjit A. Seshia,et al.  Towards Verified Artificial Intelligence , 2016, ArXiv.

[13]  Gireeja Ranade,et al.  Verifying Controllers Against Adversarial Examples with Bayesian Optimization , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[14]  Alberto L. Sangiovanni-Vincentelli,et al.  Counterexample-Guided Data Augmentation , 2018, IJCAI.

[15]  Adam Kiezun,et al.  Grammar-based whitebox fuzzing , 2008, PLDI '08.

[16]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[17]  Germán Ros,et al.  CARLA: An Open Urban Driving Simulator , 2017, CoRL.

[18]  Mahesh Viswanathan,et al.  C2E2: A Verification Tool for Stateflow Models , 2015, TACAS.

[19]  Somesh Jha,et al.  Semantic Adversarial Deep Learning , 2018, IEEE Design & Test.

[20]  Thomas A. Henzinger,et al.  Logics and Models of Real Time: A Survey , 1991, REX Workshop.

[21]  Alberto L. Sangiovanni-Vincentelli,et al.  Scenic: Language-Based Scene Generation , 2018, ArXiv.