PIP: A Database System for Great and Small Expectations

Estimation via sampling out of highly selective join queries is well known to be problematic, most notably in online aggregation. Without goal-directed sampling strategies, samples falling outside of the selection constraints lower estimation efficiency at best, and cause inaccurate estimates at worst. This problem appears in general probabilistic database systems, where query processing is tightly coupled with sampling. By committing to a set of samples before evaluating the query, the engine wastes effort on samples that will be discarded, query processing that may need to be repeated, or unnecessarily large numbers of samples. We describe PIP, a general probabilistic database system that uses symbolic representations of probabilistic data to defer computation of expectations, moments, and other statistical measures until the expression to be measured is fully known. This approach is sufficiently general to admit both continuous and discrete distributions. Moreover, deferring samplin...

• Type: Conference
• Year: 2010
• Where: ICDE
• Authors: Oliver Kennedy, Christoph Koch
• Keywords: Database, General Probabilistic Database, Goal-directed Sampling Strategies, ICDE 2010, Selective Join Queries