Polynomial Estimators for High Frequency Moments

We present an algorithm for computing Fp, the pth moment of an n-dimensional frequency vector of a data stream, for p > 2, to within 1 ± factors, ∈ (0, 1] with high constant probability. The space used is O(p2 −2 n1−2/p E(p, n) log(n) log(nmM)/ min(log(n), 4/p−2 )) bits, where, E(p, n) = (1 − 2/p)−1 (1 − n4(p−2) ) and is O(1) for p = 2 + Ω(1) and O(log n) for p = 2+O(1/ log(n). This improves upon the space required by current algorithms [10, 5, 2, 6] by a factor of at least O( −4/p min(log(n), 4/p−2 )). The update time is O((log n)(log log n)2 ). We use a new technique for designing estimators for functions of the form ψ(E [X]), where, X is a random variable and ψ is a smooth function, based on a low-degree Taylor polynomial expansion of ψ(E [X]) around an estimate of E [X].

• Type: Journal
• Year: 2011
• Where: CORR
• Authors: Sumit Ganguly
• Keywords: CORR 2011, Education, N-dimensional Frequency Vector, Pth Moment, Taylor Polynomial Expansion