A Pareto model for OLAP view size estimation

On Line Analytical Processing (OLAP) aims at gaining useful information quickly from large amounts of data residing in a data warehouse. To improve the quickness of response to queries, preaggregation is a useful strategy. However, it is usually impossible to pre-aggregate along all combinations of the dimensions. The multidimensional aspects of the data lead to combinatorial explosion in the number and potential storage size of the aggregates. We must selectively pre-aggregate. Cost/benefit analysis involves estimating the storage requirements of the aggregates in question. We present an original algorithm for estimating the number of rows in an aggregate based on the Pareto distribution model. We test the Pareto Model Algorithm empirically against three published algorithms, and conclude the Pareto Model Algorithm is consistently the best of these algorithms for estimating view size. 1 Motivation Accumulation of data in industry and organizations has led to large archives of data in...

• Type: Conference
• Year: 2001
• Where: CASCON
• Authors: Thomas P. Nadeau, Toby J. Teorey
• Keywords: CASCON 2001, CASCON 2007, Data Warehouses, Fact Table, Pareto Model Algorithm