Navid Emamdoost (University of Minnesota), Vaibhav Sharma (University of Minnesota), Taejoon Byun (University of Minnesota), Stephen McCamant (University of Minnesota)

Good tests are important in software development, but it can be hard to tell whether tests will reveal future faults that are themselves unknown. Mutation analysis, which checks whether tests reveal inserted changes in a program, is a strong measure of test suite adequacy, but common source- or compilerlevelapproachestomutationtestingarenotapplicabletosoftware available only in binary form. We explore mutation analysis as an application of the reassembleable disassembly approach to binary rewriting, building a tool for x86 binaries on top of the previously-developed Uroboros system, and apply it to the C benchmarks from SPEC CPU 2006 and to five examples of embedded control software. The results demonstrate that our approach works effectively across these software domains: as expected, tests designed for performance benchmarking reveal fewer mutants than tests generated to achieve high code coverage, but mutation scores indicate differences in test origins and features such as code size and fault-tolerance. Our binary-level tool also achieves comparable results to source-level mutation analysis despite supporting a more limited set of mutation operators. More generally we also argue that our experience shows how reassembleable disassembly is a valuable approach for constructing novel binary rewriting tools.

View More Papers

Efficient Normalized Reduction and Generation of Equivalent Multivariate Binary...

Arnau Gàmez-Montolio (City, University of London; Activision Research), Enric Florit (Universitat de Barcelona), Martin Brain (City, University of London), Jacob M. Howe (City, University of London)

Read More

Similarity Metric Method for Binary Basic Blocks of Cross-Instruction...

Xiaochuan Zhang (Artificial Intelligence Research Center, National Innovation Institute of Defense Technology), Wenjie Sun (State Key Laboratory of Mathematical Engineering and Advanced Computing), Jianmin Pang (State Key Laboratory of Mathematical Engineering and Advanced Computing), Fudong Liu (State Key Laboratory of Mathematical Engineering and Advanced Computing), Zhen Ma (State Key Laboratory of Mathematical Engineering and Advanced…

Read More

The Inconvenient Truths of Ground Truth for Binary Analysis

Jim Alves-Foss, Varsha Venugopal (University of Idaho)

Read More

All things Binary

Dr. Sergey Bratus, DARPA PI and Research Associate Professor at Dartmouth College

Read More