Filtering and Refinement: A Two-Stage Approach for Efficient and Effective Anomaly Detection

Anomaly detection is an important data mining task. Most existing methods treat anomalies as inconsistencies and spend the majority amount of time on modeling normal instances. A recently proposed, sampling-based approach may substantially boost the efficiency in anomaly detection but may also lead to weaker accuracy and robustness. In this study, we propose a two-stage approach to find anomalies in complex datasets with high accuracy as well as low time complexity and space cost. Instead of analyzing normal instances, our algorithm first employs an efficient deterministic space partition algorithm to eliminate obvious normal instances and generates a small set of anomaly candidates with a single scan of the dataset. It then checks each candidate with densitybased multiple criteria to determine the final results. This twostage framework also detects anomalies of different notions. Our experiments show that this new approach finds anomalies successfully in different conditions and ensur...

• Type: Journal
• Year: 2009
• Where: ICDM
• Authors: Xiao Yu, Lu An Tang, Jiawei Han
• Keywords: Anomaly Detection, Data Mining, ICDM 2009, Normal Instances, Obvious Normal Instances