Tracking set-expression cardinalities over continuous update streams

There is growing interest in algorithms for processing and querying continuous data streams (i.e., data that is seen only once in a fixed order) with limited memory resources. In its most general form, a data stream is actually an update stream, i.e., comprising data-item deletions as well as insertions. Such massive update streams arise naturally in several application domains (e.g., monitoring of large IP network installations, or processing of retail-chain transactions). Estimating the cardinality of set expressions defined over several (perhaps, distributed) update streams is perhaps one of the most fundamental query classes of interest; as an example, such a query may ask "what is the number of distinct IP source addresses seen in passing packets from both router R1 and R2 but not router R3?". Earlier work has only addressed very restricted forms of this problem, focusing solely on the special case of insertonly streams and specific operators (e.g., union). In this paper...

• Type: Conference
• Year: 2004
• Where: VLDB
• Authors: Sumit Ganguly, Minos N. Garofalakis, Rajeev Rastogi
• Keywords: Continuous Update Streams, Database, Massive Update Streams, Update Streams, VLDB 2004