Exploiting shared correlations in probabilistic databases

There has been a recent surge in work in probabilistic databases, propelled in large part by the huge increase in noisy data sources -from sensor data, experimental data, data from uncurated sources, and many others. There is a growing need for database management systems that can efficiently represent and query such data. In this work, we show how data characteristics can be leveraged to make the query evaluation process more efficient. In particular, we exploit what we refer to as shared correlations where the same uncertainties and correlations occur repeatedly in the data. Shared correlations occur mainly due to two reasons: (1) Uncertainty and correlations usually come from general statistics and rarely vary on a tuple-to-tuple basis; (2) The query evaluation procedure itself tends to re-introduce the same correlations. Prior work has shown that the query evaluation problem on probabilistic databases is equivalent to a probabilistic inference problem on an appropriately construct...

• Type: Journal
• Year: 2008
• Where: PVLDB
• Authors: Prithviraj Sen, Amol Deshpande, Lise Getoor
• Keywords: Correlations, Probabilistic Databases, PVLDB 2008, Query Evaluation