The early bird gets the buzz: detecting anomalies and emerging trends in information networks

In this work we propose a novel approach to anomaly detection in streaming communication data. We first build a stochastic model for the system based on temporal communication patterns across each edge, which we call the REWARDS (REneWal theory Approach for Real-time Data Streams) model. We then define a measure of anomaly for an arbitrary subgraph based on the likelihood of its recent activity given past behavior. Finally, we develop an algorithm to efficiently identify subgraphs with the most anomalous activity. Although our work has until now focused on the cybersecurity domain, the model we present is more broadly applicable to information retrieval in data streams and information networks. Categories and Subject Descriptors E.1 [Data Structures]: Graphs and networks; G.2.2 [Graph Theory]: Graph algorithms; G.3 [Probability and Statistics]: Time series analysis; G.3 [Probability and Statistics]: Renewal theory General Terms Algorithms, Design, Experimentation, Performance

• Type: Journal
• Year: 2012
• Where: WSDM
• Authors: Brian Thompson
• Keywords: Anomaly Detection, Data Mining, Probability And Statistics, Real Time Data, WSDM 2012