More Web Proxy on the site http://driver.im/

Article

Data vaults: a symbiosis between database technology and scientific file repositories

Authors:

Milena Ivanova,

Martin Kersten,

Stefan ManegoldAuthors Info & Claims

SSDBM'12: Proceedings of the 24th international conference on Scientific and Statistical Database Management

Pages 485 - 494

https://doi.org/10.1007/978-3-642-31235-9_32

Published: 25 June 2012 Publication History

Abstract

In this short paper we outline the data vault, a database-attached external file repository. It provides a true symbiosis between a DBMS and existing file-based repositories. Data is kept in its original format while scalable processing functionality is provided through the DBMS facilities. In particular, it provides transparent access to all data kept in the repository through an (array-based) query language using the file-type specific scientific libraries.

The design space for data vaults is characterized by requirements coming from various fields. We present a reference architecture for their realization in (commercial) DBMSs and a concrete implementation in MonetDB for remote sensing data geared at content-based image retrieval.

References

[1]

Alagiannis, I., Borovica, R., Branco, M., Idreos, S., Ailamaki, A.: NoDB: Efficient Query Execution on Raw Data Files. In: SIGMOD (2012).

Digital Library

[2]

Baumann, P.: Large-Scale Earth Science Services: A Case for Databases. In: ER (Workshops), pp. 75-84 (2006).

Digital Library

[3]

Baumann, P., et al.: The multidimensional database system RasDaMan. SIGMOD Rec. 27(2), 575-577 (1998).

Digital Library

[4]

Cerra, D., Datcu, M.: Image Retrieval using Compression-based Techniques. In: International ITG Conference on Source and Channel Coding (2010).

[5]

Dumitru, C.O., Molina, D.E., et al.: TELEIOS WP3: KDD concepts and methods proposal: report and design recommendations, http://www.earthobservatory.eu/ deliverables/FP7-257662-TELEIOS-D3.1.pdf

[6]

FITS. Flexible Image Transport System, http://heasarc.nasa.gov/docs/heasarc/fits.html

[7]

GeoTIFF, http://trac.osgeo.org/geotiff/

[8]

Hey, T., Tansley, S., Tolle, K.: The Fourth Paradigm: Data-Intensive Scientific Discovery. Microsoft Research (2009).

[9]

Ivanova, M., Kersten, M., Nes, N., Gonçalves, R.: An Architecture for Recycling Intermediates in a Column-store. ACM Trans. Database Syst. 35(4), 24 (2010).

Digital Library

[10]

Kunchithapadam, K., Zhang, W., et al.: Oracle Database Filesystem. In: SIGMOD, pp. 1149-1160 (2011).

Digital Library

[11]

MonetDB (2012), http://www.monetdb.org/

[12]

Oracle. Oracle Spatial GeoRaster Developer's Guide, 11g Release 2 (11.2).

[13]

PostGIS, http://www.postgis.org/

[14]

SEED. Standard for the exchange of earthquake data (May 2010), http://www.iris.edu/manuals/SEEDManual_V2.4.pdf

[15]

SQL/MED. ISO/IEC 9075-9:2008 Information technology - Database languages - SQL - Part 9: Management of External Data (SQL/MED).

[16]

Stolte, E., von Praun, C., Alonso, G., Gross, T.R.: Scientific data repositories: Designing for a moving target. In: SIGMOD Conference, pp. 349-360 (2003).

Digital Library

[17]

Zhang, Y., Kersten, M., Ivanova, M., Nes, N.: SciQL: Bridging the Gap between Science and Relational DBMS. In: IDEAS, pp. 124-133 (2011).

Digital Library

Cited By

Fang YZou CChien A(2019)Accelerating raw data analysis with the ACCORDA software and hardware architectureProceedings of the VLDB Endowment10.14778/3342263.334263412:11(1568-1582)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.14778/3342263.3342634
Zhao WRusu FDong BWu KHo ANugent PSacharidis DGamper JBöhlen M(2018)Distributed caching for processing raw arraysProceedings of the 30th International Conference on Scientific and Statistical Database Management10.1145/3221269.3221295(1-12)Online publication date: 9-Jul-2018
https://dl.acm.org/doi/10.1145/3221269.3221295
Azim TKarpathiotakis MAilamaki A(2017)ReCacheProceedings of the VLDB Endowment10.14778/3157794.315780111:3(324-337)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.14778/3157794.3157801
Show More Cited By

Recommendations

Data Vaults: Database Technology for Scientific File Repositories

Current data-management systems and analysis tools fail to meet scientists' data-intensive needs. A "data vault" approach lets researchers effectively and efficiently explore and analyze information.
Data vaults: a database welcome to scientific file repositories
SSDBM '13: Proceedings of the 25th International Conference on Scientific and Statistical Database Management

Efficient management and exploration of high-volume scientific file repositories have become pivotal for advancement in science. We propose to demonstrate the Data Vault, an extension of the database system architecture that transparently opens ...
Nested bitemporal relational data model

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

SSDBM'12: Proceedings of the 24th international conference on Scientific and Statistical Database Management

June 2012

653 pages

ISBN:9783642312342

Editors:
Anastasia Ailamaki
Computer Science, EPFL IC SIN-GE, Ecole Polytechnique Federale de Lausanne, Lausanne, Switzerland
,
Shawn Bowers
Department of Computer Science, Gonzaga University, Spokane, WA, Switzerland

Sponsors

Univ. of Athens: University of Athens
Piraeus Bank: Bank of Piraeus

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 25 June 2012

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 17 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Fang YZou CChien A(2019)Accelerating raw data analysis with the ACCORDA software and hardware architectureProceedings of the VLDB Endowment10.14778/3342263.334263412:11(1568-1582)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.14778/3342263.3342634
Zhao WRusu FDong BWu KHo ANugent PSacharidis DGamper JBöhlen M(2018)Distributed caching for processing raw arraysProceedings of the 30th International Conference on Scientific and Statistical Database Management10.1145/3221269.3221295(1-12)Online publication date: 9-Jul-2018
https://dl.acm.org/doi/10.1145/3221269.3221295
Azim TKarpathiotakis MAilamaki A(2017)ReCacheProceedings of the VLDB Endowment10.14778/3157794.315780111:3(324-337)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.14778/3157794.3157801
Olma MKarpathiotakis MAlagiannis IAthanassoulis MAilamaki A(2017)SlalomProceedings of the VLDB Endowment10.14778/3115404.311541510:10(1106-1117)Online publication date: 1-Jun-2017
https://dl.acm.org/doi/10.14778/3115404.3115415
Cheng YZhao WRusu FChoudhary AWu KDong B(2017)Bi-Level Online Aggregation on Raw DataProceedings of the 29th International Conference on Scientific and Statistical Database Management10.1145/3085504.3085514(1-12)Online publication date: 27-Jun-2017
https://dl.acm.org/doi/10.1145/3085504.3085514
Anagnostou AOlma MAilamaki AChirkova RYang JSuciu D(2017)AlpineProceedings of the 2017 ACM International Conference on Management of Data10.1145/3035918.3058743(1651-1654)Online publication date: 9-May-2017
https://dl.acm.org/doi/10.1145/3035918.3058743
Kyzirakos KAlvanaki FKersten M(2016)In memory processing of massive point clouds for multi-core systemsProceedings of the 12th International Workshop on Data Management on New Hardware10.1145/2933349.2933356(1-10)Online publication date: 26-Jun-2016
https://dl.acm.org/doi/10.1145/2933349.2933356
Cheng YRusu F(2015)SCANRAWACM Transactions on Database Systems10.1145/281818140:3(1-45)Online publication date: 23-Oct-2015
https://dl.acm.org/doi/10.1145/2818181
Zhao WCheng YRusu F(2015)Vertical partitioning for query processing over raw dataProceedings of the 27th International Conference on Scientific and Statistical Database Management10.1145/2791347.2791369(1-12)Online publication date: 29-Jun-2015
https://dl.acm.org/doi/10.1145/2791347.2791369
Hong ZYu CXia RXiao JWang JSun JCui C(2015)AQUAdexProceedings, Part II, of the 15th International Conference on Algorithms and Architectures for Parallel Processing - Volume 952910.1007/978-3-319-27122-4_7(92-105)Online publication date: 18-Nov-2015
https://dl.acm.org/doi/10.1007/978-3-319-27122-4_7
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Table of Contents