Transparently Gathering Provenance with Provenance Aware Condor

We observed that the Condor batch execution system exposes a lot of information about the jobs that run in the system. This observation led us to explore whether this system information could be used for provenance. The result of our explorations is Provenance Aware Condor (PAC), a system that transparently gathers provenance while jobs run in Condor. Transparent provenance gathering requires that the application not be altered in order to run in the provenance system. This requirement allows any application that can run in Condor to also run in PAC. Through SQL queries, PAC is able to answer a wide range of questions about the files used by a job and the machines that execute jobs.

• Type: Journal
• Year: 2009
• Where: FAST
• Authors: Christine F. Reilly, Jeffrey F. Naughton
• Keywords: Condor, FAST 2009, Operating System, Provenance, Provenance Aware Condor