Yue Liu (Southeast University), Zexiang Zhang (National University of Defense Technology), Jiaxun Zhu (Zhejiang University), Hao Zheng (Independent Researcher), Jiaqing Huang (Independent Researcher), Wenbo Shen (Zhejiang University), Gaoning Pan (Hangzhou Dianzi University), Yuliang Lu (National University of Defense Technology), Min Zhang (National University of Defense Technology), Zulie Pan (National University of Defense Technology), Guang Cheng (Southeast University)
VMware ESXi is a widely deployed enterprise-grade Type-1 hypervisor that serves as the foundation for modern cloud infrastructure. To reinforce privilege isolation, ESXi introduced a mandatory access control mechanism in VMKernel. However, due to VMKernel’s proprietary and closed-source nature, its internal access control architecture remains largely opaque and underexplored. Prior research has focused primarily on virtual device vulnerabilities and virtual machine escape, leaving the internal access control mechanisms and privilege model of VMKernel largely unexamined.
To address this gap, we conduct the first comprehensive security analysis of VMKernel’s access control mechanism. We develop a domain-control structure oriented analysis method to reconstruct key internal permission logic, and design a structure-aware debugging framework to support fine-grained runtime validation. Using this framework, we uncover several critical design flaws, including writable and unprotected in-memory control structures and exploitable developer-reserved syscall interfaces. We demonstrate three practical attack scenarios that abuse these flaws to bypass sandbox restrictions, escalate privileges, and gain persistent access. In total, we discovered and reported 14 vulnerabilities to VMware, all of which have been confirmed and fixed, with a total of $42,000 in bug bounties awarded.