Dealing with Disaster: Surviving Misbehaved Kernel Extensions

Today's extensible operating systems allow applications to modify kernel behavior by providing mechanisms for application code to run in the kernel address space. The advantage of this approach is that it provides improved application flexibility and performance; the disadvantage is that buggy or malicious code can jeopardize the integrity of the kernel. It has been demonstrated that it is feasible to use safe languages, software fault isolation, or virtual memory protection to safeguard the main kernel. However, such protection mechanisms do not address the full range of problems, such as resource hoarding, that can arise when application code is introduced into the kernel. In this paper, we present an analysis of extension mechanisms in the VINO kernel. VINO uses software fault isolation as its safety mechanism and a lightweight transaction system to cope with resource-hoarding. We explain how these two mechanisms are sufficient to protect against a large class of errant or mal...

• Type: Conference
• Year: 1996
• Where: OSDI
• Authors: Margo I. Seltzer, Yasuhiro Endo, Christopher Small, Keith A. Smith
• Keywords: Application Code, Kernel Address Space, Operating System, OSDI 1996, Software Fault Isolation