Real-time, load-adaptive processing of continuous queries over data streams

We introduce a new type of query, called a real-time continuous query (RCQ), that captures the real-time requirements of processing data streams. We develop techniques to efficiently process the RCQs in the presence of fluctuating query load and data load. We show that Rate-Monotonic scheduling is applicable to this problem domain, and show how to make this method adaptive to varying load conditions. When a set of queries becomes unschedulable due to load variations, we perform controlled input load shedding by dropping tuples using a novel feedback-based approach to decide which tuples to drop. Our work shows how to provide response time guarantees for processing RCQs, and enables making the appropriate trade-off between penalty due to missed deadlines and result accuracy. Our experiments show that our approach works very well and is usable in practice. Categories and Subject Descriptors H.4 [Information Systems Applications]: Miscellaneous General Terms Algorithms, Experimentation K...

• Type: Conference
• Year: 2008
• Where: DEBS
• Authors: Dhananjay Kulkarni, Chinya V. Ravishankar, Mitch Cherniack
• Keywords: DEBS 2008, Distributed And Parallel Computing, Load Shedding, Query Load, Real-time Continuous Query