Counting at Large: Efficient Cardinality Estimation in Internet-Scale Data Networks

Counting in general, and estimating the cardinality of (multi-) sets in particular, is highly desirable for a large variety of applications, representing a foundational block for the efficient deployment and access of emerging internetscale information systems. Examples of such applications range from optimizing query access plans in internet-scale databases, to evaluating the significance (rank/score) of various data items in information retrieval applications. The key constraints that any acceptable solution must satisfy are: (i) efficiency: the number of nodes that need be contacted for counting purposes must be small in order to enjoy small latency and bandwidth requirements; (ii) scalability, seemingly contradicting the efficiency goal: arbitrarily large numbers of nodes nay need to add elements to a (multi-) set, which dictates the need for a highly distributed solution, avoiding server-based scalability, bottleneck, and availability problems; (iii) access and storage load balan...

• Type: Conference
• Year: 2006
• Where: ICDE
• Authors: Nikos Ntarmos, Peter Triantafillou, Gerhard Weikum
• Keywords: Accurate Cardinality Estimation, Database, ICDE 2006, Nodes Nay Need, Set Cardinality Estimator