Angora: Efficient Fuzzing by Principled Search

Fuzzing is a popular technique for finding software bugs. However, the performance of the state-of-the-art fuzzers leaves a lot to be desired. Fuzzers based on symbolic execution produce quality inputs but run slow, while fuzzers based on random mutation run fast but have difficulty producing quality inputs. We propose Angora, a new mutation-based fuzzer that outperforms the state-of-the-art fuzzers by a wide margin. The main goal of Angora is to increase branch coverage by solving path constraints without symbolic execution. To solve path constraints efficiently, we introduce several key techniques: scalable byte-level taint tracking, context-sensitive branch count, search based on gradient descent, and input length exploration. On the LAVA-M data set, Angora found almost all the injected bugs, found more bugs than any other fuzzer that we compared with, and found eight times as many bugs as the second-best fuzzer in the program who. Angora also found 103 bugs that the LAVA authors injected but could not trigger. We also tested Angora on eight popular, mature open source programs. Angora found 6, 52, 29, 40 and 48 new bugs in file, jhead, nm, objdump and size, respectively. We measured the coverage of Angora and evaluated how its key techniques contribute to its impressive performance.

[1]  Xiangyu Zhang,et al.  Convicting exploitable software vulnerabilities: An efficient input provenance based approach , 2008, 2008 IEEE International Conference on Dependable Systems and Networks With FTCS and DCC (DSN).

[2]  Xiangyu Zhang,et al.  Efficient forward computation of dynamic slices using reduced ordered binary decision diagrams , 2004, Proceedings. 26th International Conference on Software Engineering.

[3]  Martin C. Rinard,et al.  Taint-based directed whitebox fuzzing , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[4]  Roland Groz,et al.  A Taint Based Approach for Smart Fuzzing , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[5]  Vikram S. Adve,et al.  LLVM: a compilation framework for lifelong program analysis & transformation , 2004, International Symposium on Code Generation and Optimization, 2004. CGO 2004..

[6]  Patrice Godefroid,et al.  Automated Whitebox Fuzz Testing , 2008, NDSS.

[7]  Dawson R. Engler,et al.  KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs , 2008, OSDI.

[8]  Christopher Krügel,et al.  Driller: Augmenting Fuzzing Through Selective Symbolic Execution , 2016, NDSS.

[9]  Koushik Sen DART: Directed Automated Random Testing , 2009, Haifa Verification Conference.

[10]  David Brumley,et al.  Unleashing Mayhem on Binary Code , 2012, 2012 IEEE Symposium on Security and Privacy.

[11]  David Leon,et al.  Detecting and debugging insecure information flows , 2004, 15th International Symposium on Software Reliability Engineering.

[12]  Koushik Sen,et al.  Symbolic execution for software testing: three decades later , 2013, CACM.

[13]  Abhik Roychoudhury,et al.  Coverage-Based Greybox Fuzzing as Markov Chain , 2017, IEEE Trans. Software Eng..

[14]  William K. Robertson,et al.  LAVA: Large-Scale Automated Vulnerability Addition , 2016, 2016 IEEE Symposium on Security and Privacy (SP).

[15]  Christopher Krügel,et al.  SOK: (State of) The Art of War: Offensive Techniques in Binary Analysis , 2016, 2016 IEEE Symposium on Security and Privacy (SP).

[16]  Herbert Bos,et al.  VUzzer: Application-aware Evolutionary Fuzzing , 2017, NDSS.

[17]  Herbert Bos,et al.  Dowsing for Overflows: A Guided Fuzzer to Find Buffer Boundary Violations , 2013, USENIX Security Symposium.

[18]  Herbert Bos,et al.  Argos: an emulator for fingerprinting zero-day attacks for advertised honeypots with automatic signature generation , 2006, EuroSys.

[19]  Yang Liu,et al.  Steelix: program-state based binary fuzzing , 2017, ESEC/SIGSOFT FSE.

[20]  Aarti Gupta,et al.  DTAM: dynamic taint analysis of multi-threaded programs for relevancy , 2012, SIGSOFT FSE.

[21]  David Brumley,et al.  Scheduling black-box mutational fuzzing , 2013, CCS.

[22]  David Brumley,et al.  Optimizing Seed Selection for Fuzzing , 2014, USENIX Security Symposium.

[23]  David Brumley,et al.  Program-Adaptive Mutational Fuzzing , 2015, 2015 IEEE Symposium on Security and Privacy.

[24]  Guofei Gu,et al.  TaintScope: A Checksum-Aware Directed Fuzzing Tool for Automatic Software Vulnerability Detection , 2010, 2010 IEEE Symposium on Security and Privacy.

[25]  Byung-Gon Chun,et al.  TaintDroid: An Information-Flow Tracking System for Realtime Privacy Monitoring on Smartphones , 2010, OSDI.

[26]  John C. S. Lui,et al.  TaintART: A Practical Multi-level Information-Flow Tracking System for Android RunTime , 2016, CCS.