Network-Based Problem Detection for Distributed Systems

We introduce a network-based problem detection framework for distributed systems, which includes a data-mining method for discovering dynamic dependencies among distributed services from transaction data collected from network, and a novel problem detection method based on the discovered dependencies. From observed containments of transaction execution time periods, we estimate the probabilities of accidental and non-accidental containments, and build a competitive model for discovering direct dependencies by using a model estimation method based on the online EM algorithm. Utilizing the discovered dependency information, we also propose a hierarchical problem detection framework, where microscopic dependency information is incorporated with a macroscopic anomaly metric that monitors the behavior of the system as a whole. This feature is made possible by employing a network-based design which provides overall information of the system without any impact on the performance.

• Type: Conference
• Year: 2005
• Where: ICDE
• Authors: Hisashi Kashima, Tadashi Tsumura, Tsuyoshi Idé, Takahide Nogayama, Ryo Hirade, Hiroaki Etoh, Takeshi Fukuda
• Keywords: Database, ICDE 2005, Network-based Problem Detection, Problem Detection Framework, Problem Detection Method