Operating System Interface Obfuscation and the Revealing of Hidden Operations

Many software security solutions—including malware analyzers, information flow tracking systems, auditing utilities, and host-based intrusion detectors—rely on knowledge of standard system call interfaces to reason about process execution behavior. In this work, we first obfuscate the Windows and Linux system call interfaces to degrade the effectiveness of these tools. Our attack, called Illusion, invokes privileged kernel operations in the kernel at the request of user-level processes without requiring those processes to call the actual system calls corresponding to the operations. The Illusion interface hides system operations from user-, kernel-, and hypervisor-level monitors mediating the conventional system-call interface. Illusion alters neither static kernel code nor read-only dispatch tables, remaining elusive from tools protecting kernel memory. We then consider the problem of Illusion attacks and augment system call data with kernel-level execution information to expos...

• Type: Journal
• Year: 2011
• Where: DIMVA
• Authors: Abhinav Srivastava, Andrea Lanzi, Jonathon T. Giffin, Davide Balzarotti
• Keywords: Computer Networks, DIMVA 2011, Intrusion Detectors, Kernel Memory, Software Security Solutions