Maintaining Stream Statistics over Sliding Windows

We consider the problem of maintaining aggregates and statistics over data streams, with respect to the last N data elements seen so far. We refer to this model as the sliding window model. We consider the following basic problem: Given a stream of bits, maintain a count of the number of 1's in the last N elements seen from the stream. We show that, using O( 1 log2 N) bits of memory, we can estimate the number of 1's to within a factor of 1 + . We also give a matching lower bound of ( 1 log2 N) memory bits for any deterministic or randomized algorithms. We extend our scheme to maintain the sum of the last N positive integers and provide matching upper and lower bounds for this more general problem as well. We also show how to efficiently compute the Lp norms (p [1, 2]) of vectors in the sliding window model using our techniques. Using our algorithm, one can adapt many other techniques to work for the sliding window model with a multiplicative overhead of O( 1 log N) in memor...

• Type: Journal
• Year: 2002
• Where: SIAMCOMP
• Authors: Mayur Datar, Aristides Gionis, Piotr Indyk, Rajeev Motwani
• Keywords: Data Streams, SIAMCOMP 2002, Sliding Window, Window Model