Shijia Li (College of Computer Science, NanKai University and the Tianjin Key Laboratory of Network and Data Security Technology), Chunfu Jia (College of Computer Science, NanKai University and the Tianjin Key Laboratory of Network and Data Security Technology), Pengda Qiu (College of Computer Science, NanKai University and the Tianjin Key Laboratory of Network and Data Security Technology), Qiyuan Chen (College of Computer Science, NanKai University and the Tianjin Key Laboratory of Network and Data Security Technology), Jiang Ming (University of Texas at Arlington), Debin Gao (Singapore Management University)

Code virtualization is a well-known sophisticated obfuscation technique that uses custom virtual machines (VM) to emulate the semantics of original native instructions. Commercial VM-based obfuscators (e.g., Themida and VMProtect) are often abused by malware developers to conceal malicious behaviors. Since the internal mechanism of commercial obfuscators is a black box, it is a daunting challenge for the analyst to understand the behavior of virtualized programs. To figure out the code virtualization mechanism and design deobfuscation techniques, the analyst has to perform reverse-engineering on large-scale highly obfuscated programs. This knowledge learning process suffers from painful cost and imprecision.

In this project, we study how to automatically extract knowledge from the commercial VM-based obfuscator via a novel textit{chosen-instruction attack} (CIA) technique. Our idea is inspired by chosen-plaintext attack, which is a cryptanalysis attack model to gain information that reduces the security of the encryption scheme. Given a commercial VM-based obfuscator, we carefully construct input programs, proactively interact with the obfuscator, and extract knowledge from virtualized output programs. We propose using the anchor instruction and the guided simplification technique to efficiently locate and extract knowledge-related instructions from output programs, respectively. Our experimental results demonstrate that the modern commercial VM-based obfuscators are under the threat of CIA. We have discovered 760 anchor instructions and extracted 1,915 verified instruction mapping rules from the four most widely used commercial obfuscators. The extracted knowledge enables security analysts to understand virtualized malware and improve deobfuscation techniques. Besides, we also contributed the first fine-grained benchmark suite for systematically evaluating the deobfuscation techniques. The evaluation result shows that three state-of-the-art deobfuscation techniques are insufficient to defeat modern commercial VM-based obfuscators and can be improved by our extracted knowledge.

View More Papers

Demo #5: Disclosing the Pringles Syndrome in Tesla FSD...

Zhisheng Hu (Baidu), Shengjian Guo (Baidu) and Kang Li (Baidu)

Read More

What You See is Not What the Network Infers:...

Yijun Yang (The Chinese University of Hong Kong), Ruiyuan Gao (The Chinese University of Hong Kong), Yu Li (The Chinese University of Hong Kong), Qiuxia Lai (Communication University of China), Qiang Xu (The Chinese University of Hong Kong)

Read More

FakeGuard: Exploring Haptic Response to Mitigate the Vulnerability in...

Aditya Singh Rathore (University at Buffalo, SUNY), Yijie Shen (Zhejiang University), Chenhan Xu (University at Buffalo, SUNY), Jacob Snyderman (University at Buffalo, SUNY), Jinsong Han (Zhejiang University), Fan Zhang (Zhejiang University), Zhengxiong Li (University of Colorado Denver), Feng Lin (Zhejiang University), Wenyao Xu (University at Buffalo, SUNY), Kui Ren (Zhejiang University)

Read More

datAFLow: Towards a Data-Flow-Guided Fuzzer

Adrian Herrera (Australian National University), Mathias Payer (EPFL), Antony Hosking (Australian National University)

Read More