Comprehensive Log Compression with Frequent Patterns

In this paper we present a comprehensive log compression (CLC) method that uses frequent patterns and their condensed representations to identify repetitive information from large log files generated by communications networks. We also show how the identified information can be used to separate and filter out frequently occurring events that hide other, unique or only a few times occurring events. The identification can be done without any prior knowledge about the domain or the events. For example, no pre-defined patterns or value combinations are needed. This separation makes it easier for a human observer to perceive and analyse large amounts of log data. The applicability of the CLC method is demonstrated with real-world examples from data communication networks.

• Type: Conference
• Year: 2003
• Where: DAWAK
• Authors: Kimmo Hätönen, Jean-François Boulicaut, Mika Klemettinen, Markus Miettinen, Cyrille Masson
• Keywords: Comprehensive Log Compression, Data Mining, DAWAK 2003, Large Log files, Times Occurring Events