An optimal strategy for monitoring top-k queries in streaming windows

Continuous top-k queries, which report a certain number (k) of top preferred objects from data streams, are important for a broad class of real-time applications, ranging from financial analysis to network traffic monitoring. Existing solutions for tackling this problem aim to reduce the computational costs by incrementally updating the top-k results upon each window slide. However, they all suffer from the performance bottleneck of periodically requiring a complete recomputation of the top-k results from scratch. Such an operation is not only computationally expensive but also causes significant memory consumption, as it requires keeping all objects alive in the query window. To solve this problem, we identify the "Minimal Top-K candidate set" (MTK), namely the subset of stream objects that is both necessary and sufficient for continuous top-k monitoring. Based on this theoretical foundation, we design the MinTopk algorithm that elegantly maintains MTK and thus eliminat...

• Type: Journal
• Year: 2011
• Where: EDBT
• Authors: Di Yang, Avani Shastri, Elke A. Rundensteiner, Matthew O. Ward
• Keywords: Database, EDBT 2011, Memory Consumption, Memory Utilization, Performance Bottleneck