How to barter bits for chronons: compression and bandwidth trade offs for database scans

Two trends are converging to make the CPU cost of a table scan a more important component of database performance. First, table scans are becoming a larger fraction of the query processing workload, and second, large memories and compression are making table scans CPU, rather than disk bandwidth, bound. Data warehouse systems have found that they can avoid the unpredictability of joins and indexing and achieve good performance by using massive parallel processing to perform scans over compressed vertical partitions of a denormalized schema. In this paper we present a study of how to make such scans faster by the use of a scan code generator that produces code tuned to the database schema, the compression dictionaries, the queries being evaluated and the target CPU architecture. We investigate a variety of compression formats and propose two novel optimizations: tuple length quantization and a field length lookup table, for efficiently processing variable length fields and tuples. We p...

• Type: Conference
• Year: 2007
• Where: SIGMOD
• Authors: Allison L. Holloway, Vijayshankar Raman, Garret Swart, David J. DeWitt
• Keywords: Compression Formats, Database, Length Lookup Table, SIGMOD 2007, Table Scans