demonstration

Packing experiments for sharing and publication

Authors:

Fernando Chirigati,

Dennis Shasha,

Juliana FreireAuthors Info & Claims

SIGMOD '13: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data

Pages 977 - 980

https://doi.org/10.1145/2463676.2465269

Published: 22 June 2013 Publication History

Get Access

Abstract

Reproducibility is a core component of the scientific process. Revisiting and reusing past results allow science to move forward - "standing on the shoulders of giants", as Newton once said. An impediment to the adoption of computational reproducibility is that authors find it difficult to generate a compendium that encompasses all the required components to correctly reproduce their experiments. Even when a compendium is available, reviewers and readers may have difficulties in verifying the results on platforms different from the ones where the experiments were originally run. As a step towards simplifying the process of creating reproducible experiments, we have developed ReproZip, a tool that automatically captures the provenance of experiments and packs all the necessary files, library dependencies and variables to reproduce the results. Reviewers can then unpack and run the experiments without having to install any additional software. We will demonstrate real use cases for ReproZip, how packages are created, and how reviewers can validate and explore experiments.

References

[1]

S. B. Davidson and J. Freire. Provenance and scientific workflows: challenges and opportunities. In SIGMOD, pages 1345--1350, 2008.

Digital Library

Google Scholar

[2]

A. Davison. Automated capture of experiment context for easier reproducibility in computational research. Computing in Science Engineering, 14(4):48--56, july-aug. 2012.

Digital Library

Google Scholar

[3]

D. Donoho, A. Maleki, I. Rahman, M. Shahram, and V. Stodden. Reproducible research in computational harmonic analysis. Computing in Science & Engineering, 11(1):8--18, Jan.-Feb. 2009.

Digital Library

Google Scholar

[4]

J. Freire, D. Koop, E. Santos, C. Scheidegger, C. Silva, and H. T. Vo. The Architecture of Open Source Applications, chapter VisTrails. Lulu.com, 2011.

Google Scholar

[5]

J. Freire and C. T. Silva. Making Computations and Publications Reproducible with VisTrails. Computing in Science and Engineering, 14(4):18--25, 2012.

Digital Library

Google Scholar

[6]

GenePattern. http://www.broadinstitute.org/cancer/software/genepattern/.

Google Scholar

[7]

P. Guo. CDE: A Tool for Creating Portable Experimental Software Packages. Computing in Science and Engineering, 14(4):32--35, 2012.

Digital Library

Google Scholar

[8]

P. J. Guo and M. Seltzer. Burrito: wrapping your lab notebook in computational infrastructure. In Proceedings of the 4th USENIX conference on Theory and Practice of Provenance, TaPP'12, pages 7--7, Berkeley, CA, USA, 2012. USENIX Association.

Digital Library

Google Scholar

[9]

R. LeVeque. Python tools for reproducible research on hyperbolic problems. Computing in Science & Engineering, 11(1):19--27, Jan.-Feb. 2009.

Digital Library

Google Scholar

[10]

Madagascar. http://www.ahay.org/wiki/Main_Page.

Google Scholar

[11]

MongoDB. http://www.mongodb.org/.

Google Scholar

[12]

SystemTap. http://sourceware.org/systemtap/.

Google Scholar

Cited By

View all

Costa LBarbosa SCunha J(2023)Towards an IDE for Scientific Computational Experiments2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL-HCC57772.2023.00056(290-292)Online publication date: 3-Oct-2023
https://doi.org/10.1109/VL-HCC57772.2023.00056
Reina G(2023)Can Image Data Facilitate Reproducibility of Graphics and Visualizations? Toward a Trusted Scientific PracticeIEEE Computer Graphics and Applications10.1109/MCG.2023.324181943:2(89-100)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1109/MCG.2023.3241819
Mascagni MPouchard LBaldwin SElsethagen TJha SRaju BStephan ETang LVan Dam K(2019)Computational reproducibility of scientific workflows at extreme scalesInternational Journal of High Performance Computing Applications10.1177/109434201983912433:5(763-776)Online publication date: 1-Sep-2019
https://dl.acm.org/doi/10.1177/1094342019839124
Show More Cited By

Index Terms

Packing experiments for sharing and publication
1. Information systems
  1. Information systems applications

Recommendations

ReproZip: Computational Reproducibility With Ease
SIGMOD '16: Proceedings of the 2016 International Conference on Management of Data

We present ReproZip, the recommended packaging tool for the SIGMOD Reproducibility Review. ReproZip was designed to simplify the process of making an existing computational experiment reproducible across platforms, even when the experiment was put ...
Computational reproducibility: state-of-the-art, challenges, and database research opportunities
SIGMOD '12: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data

Computational experiments have become an integral part of the scientific method, but reproducing, archiving, and querying them is still a challenge. The first barrier to a wider adoption is the fact that it is hard both for authors to derive a ...
Integrative Data Management for Reproducibility of Microscopy Experiments
The Semantic Web
Abstract
Reproducibility is a fundamental factor in every domain of science since it allows scientists to trust data and results. The scientific community is interested in the results of experiments which are reproducible, reusable and understandable. In ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

SIGMOD '13: Proceedings of the 2013 ACM SIGMOD International Conference on Management of Data

June 2013

1322 pages

ISBN:9781450320375

DOI:10.1145/2463676

General Chairs:
Kenneth Ross
Columbia University
,
Divesh Srivastava
AT&T Research
,
Program Chair:
Dimitris Papadias
HKUST

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 June 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Demonstration

Conference

SIGMOD/PODS'13

Sponsor:

SIGMOD

SIGMOD/PODS'13: International Conference on Management of Data

June 22 - 27, 2013

New York, New York, USA

Acceptance Rates

SIGMOD '13 Paper Acceptance Rate 76 of 372 submissions, 20%;

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
347
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Costa LBarbosa SCunha J(2023)Towards an IDE for Scientific Computational Experiments2023 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)10.1109/VL-HCC57772.2023.00056(290-292)Online publication date: 3-Oct-2023
https://doi.org/10.1109/VL-HCC57772.2023.00056
Reina G(2023)Can Image Data Facilitate Reproducibility of Graphics and Visualizations? Toward a Trusted Scientific PracticeIEEE Computer Graphics and Applications10.1109/MCG.2023.324181943:2(89-100)Online publication date: 1-Mar-2023
https://dl.acm.org/doi/10.1109/MCG.2023.3241819
Mascagni MPouchard LBaldwin SElsethagen TJha SRaju BStephan ETang LVan Dam K(2019)Computational reproducibility of scientific workflows at extreme scalesInternational Journal of High Performance Computing Applications10.1177/109434201983912433:5(763-776)Online publication date: 1-Sep-2019
https://dl.acm.org/doi/10.1177/1094342019839124
Koop DMattoso MFreire J(2018)Provenance in WorkflowsEncyclopedia of Database Systems10.1007/978-1-4614-8265-9_80745(2912-2916)Online publication date: 7-Dec-2018
https://doi.org/10.1007/978-1-4614-8265-9_80745
Neves VOliveira DOcaña KBraganholo VMurta L(2017)Managing Provenance of Implicit Data Flows in Scientific ExperimentsACM Transactions on Internet Technology10.1145/305337217:4(1-22)Online publication date: 18-Aug-2017
https://dl.acm.org/doi/10.1145/3053372
de Oliveira Ade Oliveira DMattoso M(2017)Clouds and Reproducibility: A Way to Go to Scientific Experiments?Cloud Computing10.1007/978-3-319-54645-2_5(127-151)Online publication date: 3-Jun-2017
https://doi.org/10.1007/978-3-319-54645-2_5
Chirigati FRampin RShasha DFreire JÖzcan FKoutrika GMadden S(2016)ReproZipProceedings of the 2016 International Conference on Management of Data10.1145/2882903.2899401(2085-2088)Online publication date: 26-Jun-2016
https://dl.acm.org/doi/10.1145/2882903.2899401
Belhajjame KZhao JGarijo DGamble MHettne KPalma RMina ECorcho OGómez-Pérez JBechhofer SKlyne GGoble C(2015)Using a suite of ontologies for preserving workflow-centric research objectsWeb Semantics: Science, Services and Agents on the World Wide Web10.1016/j.websem.2015.01.00332:C(16-42)Online publication date: 1-May-2015
https://dl.acm.org/doi/10.1016/j.websem.2015.01.003
Belhajjame KZhao JGarijo DGamble MHettne KPalma RMina ECorcho OGomez-Perez JBechhofer SKlyne GGoble C(undefined)Using a Suite of Ontologies for Preserving Workflow-Centric Research ObjectsSSRN Electronic Journal10.2139/ssrn.3199184
https://doi.org/10.2139/ssrn.3199184

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

ReproZip: Computational Reproducibility With Ease

Computational reproducibility: state-of-the-art, challenges, and database research opportunities

Integrative Data Management for Reproducibility of Microscopy Experiments