Diamond in the Rough: Finding Hierarchical Heavy Hitters in Multi-Dimensional Data

Data items archived in data warehouses or those that arrive online as streams typically have attributes which take values from multiple hierarchies (e.g., time and geographic location; source and destination IP addresses). Providing an aggregate view of such data is important to summarize, visualize, and analyze. We develop the aggregate view based on certain hierarchically organized sets of large-valued regions ("heavy hitters"). Such Hierarchical Heavy Hitters (HHHs) were previously introduced as a crucial aggregation technique in one dimension. In order to analyze the wider range of data warehousing applications and realistic IP data streams, we generalize this problem to multiple dimensions. We identify and study two variants of HHHs for multi-dimensional data, namely the "overlap" and "split" cases, depending on how an aggregate computed for a child node in the multi-dimensional hierarchy is propagated to its parent element(s). For data warehousing a...

• Type: Conference
• Year: 2004
• Where: SIGMOD
• Authors: Graham Cormode, Flip Korn, S. Muthukrishnan, Divesh Srivastava
• Keywords: Data Stream Applications, Data Warehousing Applications, Database, IP Data Streams, SIGMOD 2004