On-the-fly sharing for streamed aggregation

Data streaming systems are becoming essential for monitoring applications such as financial analysis and network intrusion detection. These systems often have to process many similar but different queries over common data. Since executing each query separately can lead to significant scalability and performance problems, it is vital to share resources by exploiting similarities in the queries. In this paper we present ways to efficiently share streaming aggregate queries with differing periodic windows and arbitrary selection predicates. A major contribution is our sharing technique that does not require any up-front multiple query optimization. This is a significant departure from existing techniques that rely on complex static analyses of fixed query workloads. Our approach is particularly vital in streaming systems where queries can join and leave the system at any point. We present a detailed performance study that evaluates our strategies with an implementation and real data. In ...

• Type: Conference
• Year: 2006
• Where: SIGMOD
• Authors: Sailesh Krishnamurthy, Chung Wu, Michael J. Franklin
• Keywords: Aggregate Queries, Data Streaming Systems, Database, SIGMOD 2006, Up-front Multiple Query