A Fast Algorithm for Approximate Quantiles in High Speed Data Streams

We present a fast algorithm for computing approximate quantiles in high speed data streams with deterministic error bounds. For data streams of size N where N is unknown in advance, our algorithm partitions the stream into sub-streams of exponentially increasing size as they arrive. For each sub-stream which has a fixed size, we compute and maintain a multi-level summary structure using a novel algorithm. In order to achieve high speed performance, the algorithm uses simple block-wise merge and sample operations. Overall, our algorithms for fixed-size streams and arbitrary-size streams have a computational cost of O(N log(1 log N)) and an average per-element update cost of O(log log N) if is fixed.

• Type: Conference
• Year: 2007
• Where: SSDBM
• Authors: Qi Zhang, Wei Wang 0010
• Keywords: Algorithms, Data Streams, Database, Speed Data Streams, SSDBM 2007