Approximate Medians and other Quantiles in One Pass and with Limited Memory

We present new algorithms for computing approximate quantiles of large datasets in a single pass. The approximation guarantees are explicit, and apply without regard to the value distribution or the arrival distributions of the dataset. The main memory requirements are smaller than those reported earlier by an order of magnitude. We also discuss methods that couple the approximation algorithms with random sampling to further reduce memory requirements. With sampling, the approximation guarantees are explicit but probabilistic, i.e., they apply with respect to a (user controlled) confidence parameter. We present the algorithms, their theoretical analysis and simulation results.

• Type: Conference
• Year: 1998
• Where: SIGMOD
• Authors: Gurmeet Singh Manku, Sridhar Rajagopalan, Bruce G. Lindsay
• Keywords: Approximation Algorithms, Approximation Guarantees, Database, Memory Requirements, SIGMOD 1998