More Web Proxy on the site http://driver.im/

Article

Scientific data repositories: designing for a moving target

Authors:

Christoph von Praun,

Gustavo Alonso,

Thomas GrossAuthors Info & Claims

SIGMOD '03: Proceedings of the 2003 ACM SIGMOD international conference on Management of data

Pages 349 - 360

https://doi.org/10.1145/872757.872800

Published: 09 June 2003 Publication History

Abstract

Managing scientific data warehouses requires constant adaptations to cope with changes in processing algorithms, computing environments, database schemas, and usage patterns. We have faced this challenge in the RHESSI Experimental Data Center (HEDC), a datacenter for the RHESSI NASA spacecraft. In this paper we describe our experience in developing HEDC and discuss in detail the design choices made. To successfully accommodate typical adaptations encountered in scientific data management systems, HEDC (i) clearly separates generic from domain specific code in all tiers, (ii) uses a file system for the actual data in combination with a DBMS to manage the corresponding meta data, and (iii) revolves around a middle tier designed to scale if more browsing or processing power is required. These design choices are valuable contributions as they address common concerns in a wide range of scientific data management systems.

References

[1]

Aladin. http://aladin.u-strasbg.fr/java/.

[2]

Astrobrowse. http://heasarc.gsfc.nasa.gov/ab/.

[3]

Virtual Obs. http://www.eso.org/projects/avo/.

[4]

ADS. http://adswww.harvard.edu/.

[5]

BANERJEE, S. A DBS Platform for Bioinformatics. In VLDB, Cairo, Egypt (Sept. 2000), pp. 705--710.

[6]

BARCLAY, T., GRAY, J., AND SLUTZ, D. Microsoft TerraServer: a spatial data warehouse. In SIGMOD, Dallas, USA (2000).

Digital Library

[7]

CIO. http://ircatalog.gsfc.nasa.gov/.

[8]

CDA. http://cdsarc.u-strasbg.fr/.

[9]

DSP. http://www.cacr.caltech.edu/digital_sky.html.

[10]

FREW, J. Data management for earth science systems. Sigmod Record 26, 1 (1997), 27--31.

Digital Library

[11]

FREYTAG, J. The future home of data. In VLDB, Hong-Kong, China (Aug. 2002).

[12]

GAMMA, E. et al. Design Patterns. Addison Wesley, 1995.

[13]

GRAY, J., AND SZALAY, A. The world wide telescope. In CACM. Vol. 45, No. 11 (Nov. 2002), pp. 50--54.

Digital Library

[14]

Hubble Space Telescope. http://hubble.nasa.gov/.

[15]

Infrared Space Obs. http://www.iso.vilspa.esa.es/.

[16]

KAESTLE, G., SHEK, E., AND DAO, S. Sharing experiences from scientific experiments. In SSDBM, Cleveland, USA (1999).

Digital Library

[17]

SAINT-HILAIRE, P., ET AL. The RHESSI Experimental Data Center. In Solar Physics 210(1--2), pp. 143--164 (Dec. 2002).

[18]

SHEIKHOLESLAMI, G., ET AL. WaveCluster: A wavelet based clustering approach for spatial data. VLDB Journal 8, 3--4 (2000), 289--304.

Digital Library

[19]

Sloan Digital Sky Survey. http://www.sdss.org/.

[20]

STOESSER, G., ET AL. The EMBL nucleotide sequence database. Nuclear Acids Research 27, 1 (1999), 18--24.

[21]

STOLTE, E., AND ALONSO, G. Efficient exploration of large scientific databases. In VLDB, Hong Kong, China (Aug 2002), pp. 622--633.

[22]

STOLTE, E., AND ALONSO, G. Optimizing scientific databases for Client-Side proccessing. In EDBT, Prague, Czech Republic (Mar 2002), pp. 390--408.

[23]

SZALAY, A. S., ET AL. Designing and mining multi-terabyte astronomy archives: the Sloan Digital Sky Survey. In SIGMOD, Dallas, USA (2000).

Digital Library

[24]

TSUR, S. Data Mining in the Bioinformatics Domain. In VLDB, Cairo, Egypt (Sept. 2000), pp. 711--714.

[25]

WANG, J. T.-L., ET AL. Pattern matching and pattern discovery in scientific, program, and document databases. In SIGMOD, San Jose, USA (1995), p. 487.

Digital Library

[26]

ZARRO, D. SOHO synoptic database. http://sohowww.nascom.nasa.gov.

Cited By

Thomer AStarks JRayburn ALenard M(2022)Maintaining Repositories, Databases, and Digital Collections in Memory Institutions: An Integrative ReviewProceedings of the Association for Information Science and Technology10.1002/pra2.75559:1(310-323)Online publication date: 14-Oct-2022
https://doi.org/10.1002/pra2.755
Kargin YKersten MManegold SPirk H(2015)The DBMS - your big data sommelier2015 IEEE 31st International Conference on Data Engineering10.1109/ICDE.2015.7113361(1119-1130)Online publication date: Apr-2015
https://doi.org/10.1109/ICDE.2015.7113361
Müller SAlonso GAmara ACsillaghy AFlinn JLevy H(2014)PydronProceedings of the 11th USENIX conference on Operating Systems Design and Implementation10.5555/2685048.2685100(645-659)Online publication date: 6-Oct-2014
https://dl.acm.org/doi/10.5555/2685048.2685100
Show More Cited By

Index Terms

Scientific data repositories: designing for a moving target
1. Human-centered computing
  1. Visualization
    1. Visualization application domains
      1. Scientific visualization
2. Information systems
  1. Data management systems
    1. Database design and models
      1. Physical data models
  2. Information systems applications

Recommendations

Advanced Techniques for Scientific Data Warehouses
ICACC '09: Proceedings of the 2009 International Conference on Advanced Computer Control

Data warehouses using a multidimensional view of data have become very popular in both business and science in recent years. Data warehouses for scientific purposes such as medicine and bio-chemistry1 pose several great challenges to existing data ...
Data, data use, and scientific inquiry: two case studies of data practices
JCDL '12: Proceedings of the 12th ACM/IEEE-CS joint conference on Digital Libraries

Data are proliferating far faster than they can be captured, managed, or stored. What types of data are most likely to be used and reused, by whom, and for what purposes? Answers to these questions will inform information policy and the design of ...
Acknowledgments in scientific publications: Presence in Spanish science and text patterns across disciplines

The acknowledgments in scientific publications are an important feature in the scholarly communication process. This research analyzes funding acknowledgment presence in scientific publications and introduces a novel approach for discovering text ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGMOD '03: Proceedings of the 2003 ACM SIGMOD international conference on Management of data

June 2003

702 pages

ISBN:158113634X

DOI:10.1145/872757

Conference Chair:
Zachary Ives
University of Pennsylvania
,
General Chair:
Yannis Papakonstantinou
University of California, San Diego
,
Program Chair:
Alon Halevy
University of Washington

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOD: ACM Special Interest Group on Management of Data

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 June 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

SIGMOD/PODS03

Sponsor:

SIGMOD

SIGMOD/PODS03: International Conference on Management of Data and Symposium on Principles Database and Systems

June 9 - 12, 2003

California, San Diego

Acceptance Rates

SIGMOD '03 Paper Acceptance Rate 53 of 342 submissions, 15%;

Overall Acceptance Rate 785 of 4,003 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
1,130
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Thomer AStarks JRayburn ALenard M(2022)Maintaining Repositories, Databases, and Digital Collections in Memory Institutions: An Integrative ReviewProceedings of the Association for Information Science and Technology10.1002/pra2.75559:1(310-323)Online publication date: 14-Oct-2022
https://doi.org/10.1002/pra2.755
Kargin YKersten MManegold SPirk H(2015)The DBMS - your big data sommelier2015 IEEE 31st International Conference on Data Engineering10.1109/ICDE.2015.7113361(1119-1130)Online publication date: Apr-2015
https://doi.org/10.1109/ICDE.2015.7113361
Müller SAlonso GAmara ACsillaghy AFlinn JLevy H(2014)PydronProceedings of the 11th USENIX conference on Operating Systems Design and Implementation10.5555/2685048.2685100(645-659)Online publication date: 6-Oct-2014
https://dl.acm.org/doi/10.5555/2685048.2685100
Kargın YChen LDong X(2013)Turning scientists into data explorersProceedings of the 2013 SIGMOD/PODS Ph.D. symposium10.1145/2483574.2483580(25-30)Online publication date: 23-Jun-2013
https://dl.acm.org/doi/10.1145/2483574.2483580
Bechini AVetrano A(2013)Management and storage of in situ oceanographic dataInformation Systems10.1016/j.is.2012.10.00438:3(351-368)Online publication date: 1-May-2013
https://dl.acm.org/doi/10.1016/j.is.2012.10.004
(2013)BibliographyComputation and Storage in the Cloud10.1016/B978-0-12-407767-6.00021-4(109-113)Online publication date: 2013
https://doi.org/10.1016/B978-0-12-407767-6.00021-4
Ivanova MKersten MManegold S(2012)Data vaultsProceedings of the 24th international conference on Scientific and Statistical Database Management10.1007/978-3-642-31235-9_32(485-494)Online publication date: 25-Jun-2012
https://dl.acm.org/doi/10.1007/978-3-642-31235-9_32
Badia ALemire D(2011)A call to armsACM SIGMOD Record10.1145/2070736.207075040:3(61-69)Online publication date: 17-Nov-2011
https://dl.acm.org/doi/10.1145/2070736.2070750
Lopes LSilva FCouto FZamite JFerreira HSousa CSilva M(2010)Epidemic marketplaceProceedings of the First international conference on Information technology in bio- and medical informatics10.5555/1885247.1885251(31-44)Online publication date: 1-Sep-2010
https://dl.acm.org/doi/10.5555/1885247.1885251
Bui HBui PFlynn PThain DHariri SKeahey K(2010)ROARSProceedings of the 19th ACM International Symposium on High Performance Distributed Computing10.1145/1851476.1851587(766-775)Online publication date: 21-Jun-2010
https://dl.acm.org/doi/10.1145/1851476.1851587
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents