Dictionary-Based Fast Transform for Text Compression

In this paper we present StarNT, a dictionary-based fast lossless text transform algorithm. With a static generic dictionary, StarNT achieves a superior compression ratio than almost all the other recent efforts based on BWT and PPM. This algorithm utilizes ternary search tree to expedite transform encoding. Experimental results show that the average compression time has improved by orders of magnitude compared with our previous algorithm LIPT and the additional time overhead it introduced to the backend compressor is unnoticeable. Based on StarNT, we propose StarZip, a domain-specific lossless text compression utility. Using domain-specific static dictionaries embedded in the system, StarZip achieves an average improvement in compression performance (in terms of BPC) of 13% over bzip2 -9, 19% over gzip -9, and 10% over PPMD.

• Type: Conference
• Year: 2003
• Where: ITCC
• Authors: Weifeng Sun, Nan Zhang 0005, Amar Mukherjee
• Keywords: ITCC 2003, Lossless Text, Lossless Text Transform, Superior Compression Ratio