Rachael Little, Dongpeng Xu (University of New Hampshire)

Software obfuscation is a form of code protection designed to hide the inner workings of a program from reverse engineering and analysis. Mixed Boolean Arithmetic (MBA) is one popular form that obscures simple arithmetic expressions via transformation to more complex equations involving both boolean and arithmetic operations. Most prior works focused on developing strong MBA at the source code or expression level; however, how many of them are resilient against compiler optimizations still remain unknown. In this work, we carefully inspect the strength of MBA obfuscation after various compiler optimizations. We embed MBA expressions from several popular datasets into C programs and examine how they appear post-compilation using the compilers GCC, Clang, and MSVC. Surprisingly, we discover a notable trend of reduction in MBA size and complexity after compiler optimization. We report our findings and discuss how MBA expressions are impacted by compiler optimizations.

View More Papers

Detecting Obfuscated Function Clones in Binaries using Machine Learning

Michael Pucher (University of Vienna), Christian Kudera (SBA Research), Georg Merzdovnik (SBA Research)

Read More

It Doesn’t Have to Be So Hard: Efficient Symbolic...

Vaibhav Sharma (University of Minnesota), Navid Emamdoost (University of Minnesota), Seonmo Kim (University of Minnesota), Stephen McCamant (University of Minnesota)

Read More

BrowserFM: A Feature Model-based Approach to Browser Fingerprint Analysis

Maxime Huyghe (Univ. Lille, Inria, CNRS, UMR 9189 CRIStAL), Clément Quinton (Univ. Lille, Inria, CNRS, UMR 9189 CRIStAL), Walter Rudametkin (Univ. Rennes, Inria, CNRS, UMR 6074 IRISA)

Read More