Functional Dependency Generation and Applications in Pay-As-You-Go Data Integration Systems

Recently, the opportunity of extracting structured data from the Web has been identified by a number of research projects. One such example is that millions of relational-style HTML tables can be extracted from the Web. Traditional data integration approaches do not scale over such corpora with hundreds of small tables in one domain. To solve this problem, previous work has proposed pay-as-you-go data integration systems to provide, with little up-front cost, base services over loosely-integrated information. One key component of such systems, which has received little attention to date, is the need for a framework to gauge and improve the quality of the integration. We propose a framework based on functional dependencies(FDs). Unlike in traditional database design, where FDs are specified as statements of truth about all possible instances of the database; in web environment, FDs are not specified over the data tables. Instead, we generate FDs by counting-based algorithms over man...

• Type: Conference
• Year: 2009
• Where: WEBDB
• Authors: Daisy Zhe Wang, Xin Luna Dong, Anish Das Sarma, Michael J. Franklin, Alon Y. Halevy
• Keywords: Data Integration, Data Integration Approaches, Internet Technology, Pay-as-you-go Data Integration, WEBDB 2009