Tracing Lineage Beyond Relational Operators

Tracing the lineage of data is an important requirement for establishing the quality and validity of data. Recently, the problem of data provenance has been increasingly addressed in database research. Earlier work has been limited to the lineage of data as it is manipulated using relational operations within an RDBMS. While this captures a very important aspect of scientific data processing, the existing work is incapable of handling the equally important, and prevalent, cases where the data is processed by non-relational operations. This is particularly common in scientific data where sophisticated processing is achieved by programs that are not part of a DBMS. The problem of tracking lineage when non-relational operators are used to process the data is particularly challenging since there is potentially no constraint on the nature of the processing. In this paper we propose a novel technique that overcomes this significant barrier and enables the tracing of lineage of data gener...

• Type: Conference
• Year: 2007
• Where: VLDB
• Authors: Mingwu Zhang, Xiangyu Zhang, Xiang Zhang, Sunil Prabhakar
• Keywords: Data Provenance, Database, Important Requirement, Scientific Data, VLDB 2007