CoTS: A Scalable Framework for Parallelizing Frequency Counting over Data Streams

Applications involving analysis of data streams have gained significant popularity and importance. Frequency counting, frequent elements and top-k queries form a class of operators that are used for a wide range of stream analysis applications. In spite of the abundance of these algorithms, all known techniques for answering data stream queries are sequential in nature. The imminent ubiquity of Chip Multi-Processor (CMP) architectures requires algorithms that can exploit the parallelism of such architectures. In this paper, we first explore the challenges in parallelizing frequent elements and top-k queries in the context of the inherent parallelism available in multi-core processors, evaluate different naive techniques for intra-operator parallelism, and summarize the insights obtained from the different parallelization efforts. Our experimental analysis of the naive designs implemented in the paper shows that intraoperator parallelism is not straightforward and requires a compl...

• Type: Conference
• Year: 2009
• Where: ICDE
• Authors: Sudipto Das, Shyam Antony, Divyakant Agrawal, Amr El Abbadi
• Keywords: Data Streams, Database, Frequent Elements, ICDE 2009, Top-k Queries