Evaluation of redundant data storage in clusters based on multi-multicast and local storage
Authors: 
Raúl H. Palacios, Cristina Rodríguez-Quintana, 
Antonio F. Díaz, Mancia Anguita, Julio OrtegaAuthors Info & Claims
Pages 576 - 590
Abstract
Cluster platforms have an important role in high performance computing (HPC). They execute cloud computing, data-intensive computing and data center applications, which are supported on distributed file systems. The implementation of data redundancy in these file systems provides a support for high availability and error tolerance. This work proposes an implementation of redundant data storage based on the storage included in the cluster nodes, instead of more expensive approaches with a dedicated storage and network, and on multi-multicast transfers, instead on unicast transfers, to perform the multiple simultaneous data diffusion required for implementing redundant data storage. The proposal applies a recently proposed congestion control scheme that adjusts the sender injection rate, taking into account control information from the receiver nodes and the storage technology available on the cluster nodes. The implementation takes full advantage of the switch diffusion hardware and of the IGMP snooping capability of current switches, which allows to multicast a packet just to the output links with receivers joined to a multicast group. It is made at the user level directly on the UDP interface. Evaluation tests with multiple simultaneous storage accesses were performed in a CentOS cluster. Test results show a more efficient use of the cluster storage. The global bandwidth improves by using hardware related to the storage (network and storage devices) more efficiently.
References
[1]
Adamson B, Macker J, Bormann C, Handley M (2009) NACK-oriented reliable multicast (NORM) transport protocol. Technical report, RFC 5740
[2]
Boru D, Kliazovich D, Granelli F, Bouvry P, Zomaya AY (2015) Energy-efficient data replication in cloud computing datacenters. Clust Comput 18(1):385---402
[3]
Braam PJ (2002) The Lustre storage architecture
[4]
Carns PH, Ligon III WB, Ross RB, Thakur R (2000) PVFS: A parallel file system for linux clusters. In: Proceedings of the 4th Annual Linux Showcase and Conference, pp 317---327. USENIX Association
[5]
Chaturvedi N, Jain DC (2012) Analysis of replication and replication algorithms in distributed system. Int J Adv Res Comput Sci Softw Eng 2(5):261---266
[6]
Chowdhury S, Fatema K (2013) Analysing TCP performance when link experiencing packet loss. Master's thesis, University of Gothenburg, Göteborg, Sweden
[7]
Díaz AF, Anguita M, Camacho HE, Nieto E, Ortega J (2013) Two-level hash/table approach for metadata management in distributed file systems. J Supercomput 64(1):144---155
[8]
Dickens PM (2010) A lightweight, high performance communication protocol for grid computing. Clust Comput 13(1):47---66
[9]
Filgueira R, Singh DE, Carretero J, Calderón A, García F (2011) Adaptive-CoMPI: enhancing MPI-based applications-performance and scalability by using adaptive compression. Int J High Perform Comput Appl 25(1):93---114
[10]
Gemmell J, Montgomery T, Speakman T, Crowcroft J (2003) The PGM reliable multicast protocol. IEEE Network 17(1):16---22
[11]
Gu Y, Grossman R (2011) Toward efficient and simplified distributed data intensive computing. IEEE Trans Parallel Distrib Syst 22(6):974---984
[12]
Gu Y, Grossman RL (2007) UDT: UDP-based data transfer for high-speed wide area networks. Comput Netw 51(7):1777---1799 (Protocols for fast, long-distance networks)
[13]
Haddad I (2006) The HAS architecture: a highly available and scalable cluster architecture for web servers. PhD thesis, Concordia University, Concordia University Libraries
[14]
Kadhum MM, Hassanein HS (2015) Congestion-aware TCP-friendly system for multimedia transmission based on UDP. Clust Comput 18(2):693---705
[15]
Kasera SK, Hjalmtusson G, Towsley DF, Kurose JF (2000) Scalable reliable multicast using multiple multicast channels. IEEE/ACM Trans Netw 8(3):294---310
[16]
Lane RG, Daniels S, Yuan X (2007) An empirical study of reliable multicast protocols over ethernet-connected networks. Perform Eval 64(3):210---228
[17]
Li D, Xu M, Liu Y, Xie X, Cui Y, Wang J, Chen G (2013) Reliable multicast in data center networks. IEEE Trans Comput 63(8):2011---2024
[18]
Li J, Veeraraghavan M (2012) A reliable message multicast transport protocol for virtual circuits. In: 4th International conference on communications, mobility, and computing
[19]
Palacios RH, Díaz AF, Anguita M, Ortega J, Rodríguez-Quintana C (2016) High-throughput multi-multicast transfers in data center networks. J Supercomput: 1---12
[20]
Palacios RH, Díaz AF, Ortega J, Rodríguez-Quintana C, Anguita M (2015) Analyzing high-throughput multicast traffic in cluster computing. In: Proceedings of the 15th International Conference on Computational and Mathematical Methods in Science and Engineering, CMMSE 6---10 Jul 2015
[21]
Rani LS, Sudhakar K, Kumar SV (2014) Distributed file systems: a survey. Int J Comput Sci Inf Technol 5(3):3716---3721
[22]
Sheng L (1999) Java native interface: programmer's guide and reference, 1st edn. Addison-Wesley Longman Publishing, Boston
[23]
Shvachko K, Kuang H, Radia S, Chansler R (2010)The hadoop distributed file system. In: Proceedings of the 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST), IEEE Computer Society MSST '10, pp 1---10, Washington, DC, USA
[24]
Thanh TD, Mohan S, Choi E, Kim SB, Kim P (2008) A taxonomy and survey on distributed file systems. In: Fourth International Conference on Networked Computing and Advanced Information Management, 2008. NCM '08, vol 1, pp 144---149
[25]
Wang J, Xu Z (2002) Cluster file systems: a case study. Futur Gener Comput Syst 18(3):373---387 (Cluster Computing)
[26]
Weil SA, Brandt SA, Miller EL, Long DDE, Maltzahn C (2006) Ceph: a scalable, high-performance distributed file system. In: Proceedings of the 7th Symposium on Operating Systems Design and Implementation OSDI '06, pp 307---320, USENIX Association, Berkeley, CA, USA
[27]
Xiong R, Luo J, Dong F (2015) Optimizing data placement in heterogeneous hadoop clusters. Clust Comput: 1---16
- Evaluation of redundant data storage in clusters based on multi-multicast and local storage
Recommendations
High-throughput multi-multicast transfers in data center networks
It is usual that the applications executed in data centers require the distribution of the same data from one node to others at various execution points and that some of them require to cope with multiple of these diffusions in parallel. Multicast-based ...
iSCSI Simulation Study of Storage System
UKSIM '08: Proceedings of the Tenth International Conference on Computer Modeling and SimulationDemands for more storage in the industry has increased beyond that the traditional storage mechanisms can supply and thus, the concept of Storage Area Networks (SANs) has evolved. This has resulted in the design of iSCSI (Internet SCSI) protocols that ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © Copyright © 2017 Springer Science+Business Media New York.
Publisher
Kluwer Academic Publishers
United States
Publication History
Published: 01 January 2017
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 13 Jan 2025