Fast and Efficient Log File Compression

Contemporary information systems are replete with log files, created in multiple places (e.g., network servers, database management systems, user monitoring applications, system services and utilities) for multiple purposes (e.g., maintenance, security issues, traffic analysis, legal requirements, software debugging, customer management, user interface usability studies). Log files in complex systems may quickly grow to huge sizes. Often, they must be kept for long periods of time. For reasons of convenience and storage economy, log files should be compressed. However, most of the available log file compression tools use general-purpose algorithms (e.g., Deflate) which do not take advantage of redundancy specific for log files. In this paper a specialized log file compression scheme is described in five variants, differing in complexity and attained compression ratios. The proposed scheme introduces a log file transform whose output is much better compressible with general-purpose algo...

• Type: Conference
• Year: 2007
• Where: ADBIS
• Authors: Przemyslaw Skibinski, Jakub Swacha
• Keywords: ADBIS 2007, Database, Log File Compression, Log Files, Original Files