A Flexible Fuzzy Expert System for Fuzzy Duplicate Elimination in Data Cleaning

Data cleaning deals with the detection and removal of errors and inconsistencies in data, gathered from distributed sources. This process is essential for drawing correct conclusions from data in decision support systems. Eliminating fuzzy duplicate records is a fundamental part of the data cleaning process. The vagueness and uncertainty involved in detecting fuzzy duplicates make it a niche, for applying fuzzy reasoning. Although uncertainty algebras like fuzzy logic are known, their applicability to the problem of duplicate elimination has remained unexplored and unclear, until today. In this paper, a novel and flexible fuzzy expert system for detection and elimination of fuzzy duplicates in the process of data cleaning is devised, which circumvents the repetitive and inconvenient task of hard-coding. Some of the crucial advantages of this approach are its flexibility, ease of use, extendibility, fast development time and efficient run time, when used in various information systems.

• Type: Conference
• Year: 2004
• Where: DEXA
• Authors: Hamid Haidarian Shahri, Ahmad Abdollahzadeh Barforoush
• Keywords: Data Cleaning, DEXA 2004, Fuzzy Duplicate Records, Fuzzy Duplicates