Holistic Aggregates in a Networked World: Distributed Tracking of Approximate Quantiles

While traditional database systems optimize for performance on one-shot queries, emerging large-scale monitoring applications require continuous tracking of complex aggregates and data-distribution summaries over collections of physically-distributed streams. Thus, effective solutions have to be simultaneously space efficient (at each remote site), communication efficient (across the underlying communication network), and provide continuous, guaranteedquality estimates. In this paper, we propose novel algorithmic solutions for the problem of continuously tracking complex holistic aggregates in such a distributed-streams setting -- our primary focus is on approximate quantile summaries, but our approach is more broadly applicable and can handle other holistic-aggregate functions (e.g., "heavy-hitters" queries). We present the first known distributed-tracking schemes for maintaining accurate quantile estimates with provable approximation guarantees, while simultaneously optimi...

• Type: Conference
• Year: 2005
• Where: SIGMOD
• Authors: Graham Cormode, Minos N. Garofalakis, S. Muthukrishnan, Rajeev Rastogi
• Keywords: Accurate Quantile Estimates, Approximate Quantile Summaries, Database, Provable Approximation Guarantees, SIGMOD 2005