HookScout: Proactive Binary-Centric Hook Detection

Abstract. In order to obtain and maintain control, kernel malware usually makes persistent control flow modifications (i.e., installing hooks). To avoid detection, malware developers have started to target function pointers in kernel data structures, especially those dynamically allocated from heaps and memory pools. Function pointer modification is stealthy and the attack surface is large; thus, this type of attacks is appealing to malware developers. In this paper, we first conduct a systematic study of this problem, and show that the attack surface is vast, with over 18, 000 function pointers (most of them long-lived) existing within the Windows kernel. Moreover, to demonstrate this threat is realistic for closedsource operating systems, we implement two new attacks for Windows by exploiting two function pointers individually. Then, we propose a new proactive hook detection technique, and develop a prototype, called HookScout. Our approach is binary-centric, and thus can generate ho...

• Type: Conference
• Year: 2010
• Where: DIMVA
• Authors: Heng Yin, Pongsin Poosankam, Steve Hanna, Dawn Xiaodong Song
• Keywords: Attack Surface, Computer Networks, DIMVA 2010, Function Pointers, Malware Developers