A Framework for Collecting Provenance in Data-Centric Scientific Workflows

Yogesh L. Simmhan, Beth Plale, Dennis Gannon

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The increasing ability for the earth sciences to sense the world around us is resulting in a growing need for datadriven applications that are under the control of data-centric workflows composed of grid- and web- services. The focus of our work is on provenance collection for these workflows, necessary to validate the workflow and to determine quality of generated data products. The challenge we address is to record uniform and usable provenance metadata that meets the domain needs while minimizing the modification burden on the service authors and the performance overhead on the workflow engine and the services. The framework, based on a loosely-coupled publish-subscribe architecture for propagating provenance activities, satisfies the needs of detailed provenance collection while a performance evaluation of a prototype finds a minimal performance overhead (in the range of 1% for an eight service workflow using 271 data products).
Original languageEnglish
Title of host publicationWeb Services, 2006. ICWS '06. International Conference on
PublisherInstitute of Electrical and Electronics Engineers (IEEE)
Pages427-436
Number of pages10
ISBN (Print)0-7695-2669-1
DOIs
Publication statusPublished - 2006
Externally publishedYes

Fingerprint Dive into the research topics of 'A Framework for Collecting Provenance in Data-Centric Scientific Workflows'. Together they form a unique fingerprint.

Cite this