Understanding Precision in Host Based Intrusion Detection

Abstract. Many host-based anomaly detection systems monitor process execution at the granularity of system calls. Other recently proposed schemes instead verify the destinations of control-flow transfers to prevent the execution of attack code. This paper formally analyzes and compares real systems based on these two anomaly detection philosophies in terms of their attack detection capabilities, and proves and disproves several intuitions. We prove that for any system-call sequence model, under the same (static or dynamic) program analysis technique, there always exists a more precise control-flow sequence based model. While hybrid approaches combining system calls and control flows intuitively seem advantageous, especially when binary analysis constructs incomplete models, we prove that they have no fundamental advantage over simpler control-flow models. Finally, we utilize the ideas in our framework to make external monitoring feasible at the precise control-flow level. Our expe...

• Type: Conference
• Year: 2007
• Where: RAID
• Authors: Monirul I. Sharif, Kapil Singh, Jonathon T. Giffin, Wenke Lee
• Keywords: Anomaly Detection, Host-based Anomaly Detection, Precise Control-flow Sequence, RAID 2007