Scaling Down Off-the-Shelf Data Compression: Backwards-Compatible Fine-Grain Mixing

—Pu and Singaravelu presented Fine-Grain Mixing, an adaptive compression system which aimed to maximize CPU and network utilization simultaneously by splitting a network stream into a mixture of compressed and uncompressed blocks. Blocks were compressed opportunistically in a send buffer; they compressed as many blocks as they could without becoming a bottleneck. They successfully utilized all available CPU and network bandwidth even on high speed connections. In addition, they noted much greater throughput than previous adaptive compression systems. Here, we take a different view of FGMixing than was taken by Pu and Singaravelu and give another explanation for its high performance: that fine-grain mixing of compressed and uncompressed blocks enables off-the-shelf compressors to scale down their degree of compression linearly with decreasing CPU usage. Exploring the scaling behavior in-depth allows us to make a variety of improvements to finegrain mixed compression: better compress...

• Type: Journal
• Year: 2012
• Where: ICDCS
• Authors: Michael Gray, Peter Peterson, Peter L. Reiher
• Keywords: Adaptive Compression, Compression Ratios, CPU Consumption, Distributed And Parallel Computing, ICDCS 2012