[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/2818869.2818884acmotherconferencesArticle/Chapter ViewAbstractPublication Pagesase-bigdataConference Proceedingsconference-collections
research-article

Cost-Effective and Reliable Cloud Storage for Big Data

Published: 07 October 2015 Publication History

Abstract

Efficiency and reliability are critical to the performance of big data computing. Hadoop is one of the most popular cloud platforms to support big data analysis. To improve data availability, the Hadoop distributed file system usually stores blocks of files in triplicate, but it incurs high storage costs and high information-leakage risks. In this paper we propose a new cost-effective approach in xHDFS: paired blocks from two machines are erasure-coded to generate redundant blocks that are intelligently managed. We evaluate that xHDFS can effectively improve the reliability of cloud storage and tolerate network or site failures with lower storage costs.

References

[1]
Shvachko, K., Kuang, H., Radia, S., and Chansler, R. 2010. The Hadoop distributed file system. In Proceedings of the 26th IEEE Symposium on Mass Storage Systems and Technologies (May. 2010).
[2]
Ghemawat, S., Gobioff, H., and Leung, S.-T. 2003. The Google file system In Proceedings of the 19th ACM Symposium on Operating Systems Principles (Oct. 2003).
[3]
HDFS RAID. http://wiki.apache.org/hadoop/HDFS-RAID
[4]
Sathiamoorthy, M., Asteris, M., Papailiopoulos, D., Dimakis, A., Vadali, R., Chen, S., and Borthakur, D. 2013. XORing elephants: novel erasure codes for big data. In Proceedings of the 39th International Conference on Very Large Data Bases (Aug. 2013).
[5]
Wikipedia. Backup. https://en.wikipedia.org/wiki/Backup
[6]
Wu, C.-H., and Wang, P.-H. 2014. Secure multi-key file-sharing for cloud storage with erasure coding. In Proceedings of IEEE International Conference on Computer, Information, and Telecommunication Systems (Jul. 2014).

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences
ASE BD&SI '15: Proceedings of the ASE BigData & SocialInformatics 2015
October 2015
381 pages
ISBN:9781450337359
DOI:10.1145/2818869
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 October 2015

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Big Data Computing
  2. Cloud File Systems
  3. Cloud Storage
  4. Erasure Codes
  5. Hadoop
  6. Reliability
  7. Remote Backup

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

ASE BD&SI '15
ASE BD&SI '15: ASE BigData & SocialInformatics 2015
October 7 - 9, 2015
Kaohsiung, Taiwan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 246
    Total Downloads
  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 01 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media