Frank Capobianco (The Pennsylvania State University), Quan Zhou (The Pennsylvania State University), Aditya Basu (The Pennsylvania State University), Trent Jaeger (The Pennsylvania State University, University of California, Riverside), Danfeng Zhang (The Pennsylvania State University, Duke University)
Correct access control enforcement is a critical foundation for data security. The reference monitor is the key component for enforcing access control, which is supposed to provide tamperproof mediation of all security-sensitive operations. Since reference monitors are often deployed in complex software handling a wide variety of operation requests, such as operating systems and server programs, a question is whether reference monitor implementations may have flaws that prevent them from achieving these requirements. In the past, automated analyses detected flaws in complete mediation. However, researchers have not yet developed methods to detect flaws that may tamper with the reference monitor, despite the many vulnerabilities found in such programs. In this paper, we develop TALISMAN, an automated analysis for detecting flaws that may tamper the execution of reference monitor implementations. At its core, TALISMAN implements a precise information flow integrity analysis to detect violations that may tamper the construction of authorization queries. TALISMAN applies a new, relaxed variant of noninterference that eliminates several spurious implicit flow violations. TALISMAN also provides a means to vet expected uses of untrusted data in authorization using endorsement. We apply TALISMAN on three reference monitor implementations used in the Linux Security Modules framework, SELinux, AppArmor, and Tomoyo, verifying 80% of the arguments in authorization queries generated by these LSMs. Using TALISMAN, we also found vulnerabilities in how pathnames are used in authorization by Tomoyo and AppArmor allowing adversaries to circumvent authorization. TALISMAN shows that tamper analysis of reference monitor implementations can automatically verify many cases and also enable the detection of critical flaws.