Yingjie Cao (Sun Yat-sen University and The Hong Kong Polytechnic University), Xiaogang Zhu (Adelaide University), Dean Sullivan (University of New Hampshire, US), Haowei Yang, Lei Xue (Sun Yat-sen University), Xian Li (Swinburne University of Technology, Australia), Chenxiong Qian (University of Hong Kong, China), Minrui Yan (Swinburne University of Technology, Australia), Xiapu Luo (The Hong Kong Polytechnic University)
Real-time operating systems (RTOS) often expose double-fetch vulnerabilities when the kernel reads the same userspace memory location multiple times without ensuring consistency between fetches. Conventional static analysis cannot inspect proprietary, commercial off-the-shelf (COTS) RTOS kernels, and dynamic heuristics, which rely on broad time-window thresholds, suffer from high false positive rates and heavy emulation overhead. To address these challenges, we present ISOLATOS, the first hardware-supported framework for detecting doublefetch bugs in COTS RTOS. By leveraging modern CPU kernelisolation features, ISOLATOS enables kernel isolation so that cross-boundary accesses can be captured by triggering page faults. ISOLATOS then records page-fault metadata on each usermemory fetch. Finally, multiple fetches in the same system call are determined as a double-fetch bug, based on the lifecycle of system calls that ISOLATOS instruments into COTS RTOS. We evaluate ISOLATOS on three widely used RTOS, including QNX, VxWorks, and seL4, and demonstrate a 79.3× reduction in runtime overhead compared to state-of-the-art emulation-based detectors. ISOLATOS also detects double-fetch bugs with lower false positive rates than other tools. Our approach uncovers 43 previously unknown vulnerabilities in COTS RTOS (41 confirmed by vendors, 2 CVEs assigned). Additionally, we have demonstrated the real-world impact of our findings in automotive systems by exploiting them.