Rachael Little, Dongpeng Xu (University of New Hampshire)
Software obfuscation is a form of code protection designed to hide the inner workings of a program from reverse engineering and analysis. Mixed Boolean Arithmetic (MBA) is one popular form that obscures simple arithmetic expressions via transformation to more complex equations involving both boolean and arithmetic operations. Most prior works focused on developing strong MBA at the source code or expression level; however, how many of them are resilient against compiler optimizations still remain unknown. In this work, we carefully inspect the strength of MBA obfuscation after various compiler optimizations. We embed MBA expressions from several popular datasets into C programs and examine how they appear post-compilation using the compilers GCC, Clang, and MSVC. Surprisingly, we discover a notable trend of reduction in MBA size and complexity after compiler optimization. We report our findings and discuss how MBA expressions are impacted by compiler optimizations.