Navid Emamdoost (University of Minnesota), Vaibhav Sharma (University of Minnesota), Taejoon Byun (University of Minnesota), Stephen McCamant (University of Minnesota)

Good tests are important in software development, but it can be hard to tell whether tests will reveal future faults that are themselves unknown. Mutation analysis, which checks whether tests reveal inserted changes in a program, is a strong measure of test suite adequacy, but common source- or compilerlevelapproachestomutationtestingarenotapplicabletosoftware available only in binary form. We explore mutation analysis as an application of the reassembleable disassembly approach to binary rewriting, building a tool for x86 binaries on top of the previously-developed Uroboros system, and apply it to the C benchmarks from SPEC CPU 2006 and to five examples of embedded control software. The results demonstrate that our approach works effectively across these software domains: as expected, tests designed for performance benchmarking reveal fewer mutants than tests generated to achieve high code coverage, but mutation scores indicate differences in test origins and features such as code size and fault-tolerance. Our binary-level tool also achieves comparable results to source-level mutation analysis despite supporting a more limited set of mutation operators. More generally we also argue that our experience shows how reassembleable disassembly is a valuable approach for constructing novel binary rewriting tools.

View More Papers

Polypyus – The Firmware Historian

Jan Friebertshauser, Florian Kosterhon, Jiska Classen, Matthias Hollick (Secure Mobile Networking Lab, TU Darmstad)

Read More

QSynth – A Program Synthesis based approach for Binary...

Robin David (Quarkslab), Luigi Coniglio (University of Trento), Mariano Ceccato (University of Verona)

Read More

The hard things about analyzing 1’s and 0’s...

Dr. David Brumley, Carnegie Mellon University - ForAllSecure

Read More

Understanding MPU Usage in Microcontroller-based Systems in the Wild

Wei Zhou, Zhouqi Jiang (School of Cyber Science and Engineering, Huazhong University of Science and Technology), Le Guan (School of Computing, University of Georgia)

Read More