article

Tracking provenance in a virtual data grid

Authors:

Yong ZhaoAuthors Info & Claims

Concurrency and Computation: Practice & Experience, Volume 20, Issue 5

Pages 565 - 575

Published: 01 April 2008 Publication History

Abstract

The virtual data model allows data sets to be described prior to, and separately from, their physical materialization. We have implemented this model in a Virtual Data Language (VDL) and associated supporting tools, which provide for both the storage, query, and retrieval of virtual data set descriptions, and the automated, on-demand materialization of virtual data sets. We use a standardized data provenance challenge exercise to illustrate the powerful queries that can be performed on the data maintained by these tools, which for a single virtual data set can include three elements: the computational procedure(s) that must be executed to materialize the data set, the runtime log(s) produced by the execution of the computation(s), and optional metadata annotation(s) that associate application semantics with data and procedures. Copyright © 2007 John Wiley & Sons, Ltd.

Cited By

View all

Pimentel JFreire JMurta LBraganholo V(2019)A Survey on Collecting, Managing, and Analyzing Provenance from ScriptsACM Computing Surveys10.1145/331195552:3(1-38)Online publication date: 18-Jun-2019
https://dl.acm.org/doi/10.1145/3311955
Pérez BRubio JSáenz-Adán C(2018)A systematic review of provenance systemsKnowledge and Information Systems10.1007/s10115-018-1164-357:3(495-543)Online publication date: 1-Dec-2018
https://dl.acm.org/doi/10.1007/s10115-018-1164-3
Song TBarcelo PShen H(2015)Provenance-Driven Data Curation Workflow AnalysisProceedings of the 2015 ACM SIGMOD on PhD Symposium10.1145/2744680.2744691(45-50)Online publication date: 31-May-2015
https://dl.acm.org/doi/10.1145/2744680.2744691
Show More Cited By

Index Terms

Tracking provenance in a virtual data grid
1. Information systems
  1. Data management systems
  2. Information storage systems
    1. Storage architectures
      1. Distributed storage
2. Theory of computation
  1. Theory and algorithms for application domains
    1. Database theory
      1. Database query languages (principles)

Recommendations

The Grid Resource Broker workflow engine
2nd International Workshop on Workflow Management and Applications in Grid Environments (WaGe2007)

Increasingly, complex scientific applications are structured in terms of workflows. These applications are usually computationally and-or data intensive and thus are well suited for execution in grid environments. Distributed, geographically spread ...
Provenance opportunities for WS-VLAM: an exploration of an e-science and an e-business approach
WORKS '11: Proceedings of the 6th workshop on Workflows in support of large-scale science

Scientific applications are frequently modeled as a workflow that is executed under the control of a workflow management system. One crucial requirement during the execution is the validation of the generated results and the traceability of the ...
Virtual data Grid middleware services for data-intensive science: Research Articles
Middleware for Grid Computing

The GriPhyN virtual data system provides a suite of components and services for data-intensive sciences that enables scientists to systematically and efficiently describe, discover, and share large-scale data and computational resources. We describe the ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Concurrency and Computation: Practice & Experience

Concurrency and Computation: Practice & Experience Volume 20, Issue 5

The First Provenance Challenge

April 2008

183 pages

ISSN:1532-0626

EISSN:1532-0634

Issue’s Table of Contents

Publisher

John Wiley and Sons Ltd.

United Kingdom

Publication History

Published: 01 April 2008

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

View all

Pimentel JFreire JMurta LBraganholo V(2019)A Survey on Collecting, Managing, and Analyzing Provenance from ScriptsACM Computing Surveys10.1145/331195552:3(1-38)Online publication date: 18-Jun-2019
https://dl.acm.org/doi/10.1145/3311955
Pérez BRubio JSáenz-Adán C(2018)A systematic review of provenance systemsKnowledge and Information Systems10.1007/s10115-018-1164-357:3(495-543)Online publication date: 1-Dec-2018
https://dl.acm.org/doi/10.1007/s10115-018-1164-3
Song TBarcelo PShen H(2015)Provenance-Driven Data Curation Workflow AnalysisProceedings of the 2015 ACM SIGMOD on PhD Symposium10.1145/2744680.2744691(45-50)Online publication date: 31-May-2015
https://dl.acm.org/doi/10.1145/2744680.2744691
Porto Fde C. Moura AGonçalves BCosta RSpaccapietra S(2012)A scientific hypothesis conceptual modelProceedings of the 2012 international conference on Advances in Conceptual Modeling10.1007/978-3-642-33999-8_13(101-110)Online publication date: 15-Oct-2012
https://dl.acm.org/doi/10.1007/978-3-642-33999-8_13
Porto FSpaccapietra S(2011)Data model for scientific models and hypothesesThe evolution of conceptual modeling10.5555/1985703.1985719(285-305)Online publication date: 1-Jan-2011
https://dl.acm.org/doi/10.5555/1985703.1985719
Gadelha LWilde MMattoso MFoster IHowe B(2011)Exploring provenance in high performance scientific computingProceedings of the first annual workshop on High performance computing meets databases10.1145/2125636.2125643(17-20)Online publication date: 13-Nov-2011
https://dl.acm.org/doi/10.1145/2125636.2125643
Gadelha Jr. LClifford BMattoso MWilde MFoster I(2011)Provenance management in SwiftFuture Generation Computer Systems10.1016/j.future.2010.05.00327:6(775-780)Online publication date: 1-Jun-2011
https://dl.acm.org/doi/10.1016/j.future.2010.05.003
Lyle JMartin A(2010)Trusted computing and provenanceProceedings of the 2nd conference on Theory and practice of provenance10.5555/1855795.1855796(1-1)Online publication date: 22-Feb-2010
https://dl.acm.org/doi/10.5555/1855795.1855796
Moreau L(2010)The Foundations for Provenance on the WebFoundations and Trends in Web Science10.1561/18000000102:2-3(99-241)Online publication date: 1-Feb-2010
https://dl.acm.org/doi/10.1561/1800000010
Vicknair CCunningham HRuth PKraft N(2010)Research issues in data provenanceProceedings of the 48th annual ACM Southeast Conference10.1145/1900008.1900037(1-4)Online publication date: 15-Apr-2010
https://dl.acm.org/doi/10.1145/1900008.1900037
Show More Cited By

Abstract

Cited By

Index Terms

Recommendations

The Grid Resource Broker workflow engine

Provenance opportunities for WS-VLAM: an exploration of an e-science and an e-business approach

Virtual data Grid middleware services for data-intensive science: Research Articles

Comments

Information

Published In

Publisher

Publication History

Author Tags

Qualifiers

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

View options

Share

Share this Publication link

Share on social media

Affiliations