Improving Mining Quality by Exploiting Data Dependency

The usefulness of the results produced by data mining methods can be critically impaired by several factors such as (1) low quality of data, including errors due to contamination, or incompleteness due to limited bandwidth for data acquisition, and (2) inadequacy of the data model for capturing complex probabilistic relationships in data. Fortunately, a wide spectrum of applications exhibit strong dependencies between data samples. For example, the readings of nearby sensors are generally correlated, and proteins interact with each other when performing crucial functions. Therefore, dependencies among data can be successfully exploited to remedy the problems mentioned above. In this paper, we propose a unified approach to improving mining quality using Markov networks as the data model to exploit local dependencies. Belief propagation is used to efficiently compute the marginal or maximum posterior probabilities, so as to clean the data, to infer missing values, or to improve the min...

• Type: Conference
• Year: 2005
• Where: PAKDD
• Authors: Fang Chu, Yizhou Wang, Carlo Zaniolo, Douglas Stott Parker Jr.
• Keywords: Data Acquisition, Data Mining, Data Mining Methods, Data Model, PAKDD 2005