More Web Proxy on the site http://driver.im/

research-article

Guaranteeing deadlines for inter-datacenter transfers

Authors:

Ming ZhangAuthors Info & Claims

EuroSys '15: Proceedings of the Tenth European Conference on Computer Systems

Article No.: 20, Pages 1 - 14

https://doi.org/10.1145/2741948.2741957

Published: 17 April 2015 Publication History

Abstract

Inter-datacenter wide area networks (inter-DC WAN) carry a significant amount of data transfers that require to be completed within certain time periods, or deadlines. However, very little work has been done to guarantee such deadlines. The crux is that the current inter-DC WAN lacks an interface for users to specify their transfer deadlines and a mechanism for provider to ensure the completion while maintaining high WAN utilization.

This paper addresses the problem by introducing a Deadline-based Network Abstraction (DNA) for inter-DC WANs. DNA allows users to explicitly specify the amount of data to be delivered and the deadline by which it has to be completed. The malleability of DNA provides flexibility in resource allocation. Based on this, we develop a system called Amoeba that implements DNA. Our simulations and testbed experiments show that Amoeba, by harnessing DNA's malleability, accommodates 15% more user requests with deadlines, while achieving 60% higher WAN utilization than prior solutions.

Supplementary Material

MP4 File (a20-sidebyside.mp4)

Download
854.39 MB

References

[1]

"Why DC-DC WAN optimization is important," http://ovum.com/2012/07/16/why-dc-dc-wan-optimization-is-important, 2012.

[2]

H. Ballani, P. Costa, T. Karagiannis, and A. Rowstron, "Towards Predictable Datacenter Networks," in SIGCOMM, 2011.

Digital Library

[3]

B. B. Chen and P. V.-B. Primet, "Scheduling deadline-constrained bulk data transfers to minimize network congestion." in CCGRID, 2007.

Digital Library

[4]

Y. Chen, S. Jain, V. K. Adhikari, Z.-L. Zhang, and K. Xu, "A First Look at Inter-Data Center Traffic Characteristics via Yahoo! Datasets," in INFOCOM, 2011.

[5]

L. R. Ford Jr and D. R. Fulkerson, "Constructing maximal dynamic flows from static flows," Operations research, vol. 6, no. 3, pp. 419--433, 1958.

Digital Library

[6]

C. Guo, G. Lu, H. Wang, S. Yang, C. Kong, P. Sun, W. Wu, and Y. Zhang, "SecondNet: A Data Center Network Virtualization Architecture with Bandwidth Guarantees," in CoNEXT, 2010.

Digital Library

[7]

C.-Y. Hong, M. Caesar, and P. Godfrey, "Finishing flows quickly with preemptive scheduling," ACM SIGCOMM Computer Communication Review, vol. 42, no. 4, pp. 127--138, 2012.

Digital Library

[8]

C.-Y. Hong, S. Kandula, R. Mahajan, M. Zhang, V. Gill, M. Nanduri, and R. Wattenhofer, "Achieving high utilization using software-driven WAN," in SIGCOMM, 2013.

Digital Library

[9]

S. Jain, A. Kumar, S. Mandal, J. Ong, L. Poutievski, A. Singh, S. Venkata, J. Wanderer, J. Zhou, M. Zhu, J. Zolla, U. Hlzle, S. Stuart, and A. Vahdat, "B4: Experience with a Globally Deployed Software Defined WAN," in SIGCOMM, 2013.

Digital Library

[10]

V. Jalaparti, H. Ballani, P. Costa, T. Karagiannis, and A. Rowstron, "Bridging the tenant-provider gap in cloud services," in SoCC, 2012.

Digital Library

[11]

C. Jayalath, J. Stephen, and P. Eugster, "From the Cloud to the Atmosphere: Running MapReduce across Datacenters," IEEE Transactions on Computers, 2007.

[12]

V. Jeyakumar, M. Alizadeh, D. Mazieres, B. Prabhakar, C. Kim, and A. Greenberg, "EyeQ: Practical Network Performance Isolation at the Edge," in NSDI, 2013.

Digital Library

[13]

S. Kandula, I. Menache, R. Schwartz, and S. R. Babbula, "Calendaring for wide area networks," in SIGCOMM, 2014.

Digital Library

[14]

L. Lamport, "The Part-time Parliament," ACM Trans. Comput. Syst., vol. 16, no. 2, pp. 133--169, May 1998.

Digital Library

[15]

N. Laoutaris, M. Sirivianos, X. Yang, and P. Rodriguez, "Inter-Datacenter Bulk Transfers with NetStitcher," in SIGCOMM, 2011.

Digital Library

[16]

H. H. Liu, S. Kandula, R. Mahajan, M. Zhang, and D. Gelernter, "Traffic engineering with forward fault correction," in SIGCOMM, 2014.

Digital Library

[17]

L. Popa, P. Yalagandula, S. Banerjee, J. C. Mogul, and Y. T. and Jose Renato Santos, "ElasticSwitch: Practical Work-Conserving Bandwidth Guarantees for Cloud Computing," in SIGCOMM, 2013.

Digital Library

[18]

B. Raghavan, K. Vishwanath, S. Ramabhadran, K. Yocum, and A. C. Snoeren, "Cloud control with distributed rate limiting," in ACM SIGCOMM Computer Communication Review, vol. 37, no. 4, pp. 337--348, 2007.

Digital Library

[19]

K. Rajah, S. Ranka, and Y. Xia, "Advance reservations and scheduling for bulk transfers in research networks," Parallel and Distributed Systems, IEEE Transactions on, vol. 20, no. 11, pp. 1682--1697, 2009.

Digital Library

[20]

H. Rodrigues, J. R. Santos, Y. Turner, P. Soares, and D. Guedes, "Gatekeeper: Supporting bandwidth guarantees for multi-tenant datacenter networks," USENIX WIOV, 2011.

Digital Library

[21]

B. Vamanan, J. Hasan, and T. Vijaykumar, "Deadline-aware datacenter tcp (d2tcp)," in SIGCOMM, 2012.

Digital Library

[22]

C. Wilson, H. Ballani, T. Karagiannis, and A. Rowtron, "Better never than late: Meeting deadlines in datacenter networks," in ACM SIGCOMM Computer Communication Review, vol. 41, no. 4, pp. 50--61, 2011.

Digital Library

[23]

D. Xie, N. Ding, Y. C. Hu, and R. Kompella, "The Only Constant is Change: Incorporating Time-Varying Network Reservations in Data Centers," in SIGCOMM, 2012.

Digital Library

[24]

D. Zats, T. Das, P. Mohan, D. Borthakur, and R. Katz, "DeTail: Reducing the flow completion time tail in datacenter networks," ACM SIGCOMM Computer Communication Review, vol. 42, no. 4, pp. 139--150, 2012.

Digital Library

Cited By

Wang JZhao GXu HQiao CHuang H(2025)A cost-efficient traffic engineering framework with various pricing schemes in cloudsComputer Networks10.1016/j.comnet.2025.111090259(111090)Online publication date: Mar-2025
https://doi.org/10.1016/j.comnet.2025.111090
Cao JGuan YQian KGao JXiao WDong JFu BCai DZhai ESekar VYu MSeneviratne AVeitch D(2024)Crux: GPU-Efficient Communication Scheduling for Deep Learning TrainingProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672239(1-15)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672239
Fatemipour BZhang ZSt-Hilaire M(2024)A Survey on Replica Transfer Optimization Schemes in Geographically Distributed Data CentersIEEE Transactions on Network and Service Management10.1109/TNSM.2024.343716521:6(6301-6317)Online publication date: Dec-2024
https://doi.org/10.1109/TNSM.2024.3437165
Show More Cited By

Index Terms

Guaranteeing deadlines for inter-datacenter transfers

Recommendations

Guaranteeing Deadlines for Inter-Data Center Transfers

Inter-data center wide area networks inter-DC WANs carry a significant amount of data transfers that require to be completed within certain time periods, or deadlines. However, very little work has been done to guarantee such deadlines. The crux is that ...
Inter-datacenter bulk transfers with netstitcher
SIGCOMM '11: Proceedings of the ACM SIGCOMM 2011 conference

Large datacenter operators with sites at multiple locations dimension their key resources according to the peak demand of the geographic area that each site covers. The demand of specific areas follows strong diurnal patterns with high peak to valley ...
Cost-Efficient Scheduling of Bulk Transfers in Inter-Datacenter WANs

With the quick growth of traffic between data centers, inefficient transfer scheduling in inter-datacenter networks can lead to a huge waste of bandwidth thus significant bandwidth cost. Previous work have explored different ways, such as software-...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

EuroSys '15: Proceedings of the Tenth European Conference on Computer Systems

April 2015

503 pages

ISBN:9781450332385

DOI:10.1145/2741948

General Chair:
Laurent Réveillère
LaBRI, University of Bordeaux, France
,
Program Chairs:
Tim Harris
Oracle Labs, UK
,
Maurice Herlihy
Brown University

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 April 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

MSIP, Rep. of Korea

Conference

EuroSys '15

Sponsor:

SIGOPS

EuroSys '15: Tenth EuroSys Conference 2015

April 21 - 24, 2015

Bordeaux, France

Acceptance Rates

Overall Acceptance Rate 241 of 1,308 submissions, 18%

Upcoming Conference

EuroSys '25

Sponsor:
sigops

Twentieth European Conference on Computer Systems

March 30 - April 3, 2025

Rotterdam , Netherlands

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

41
Total Citations
View Citations
923
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)5

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang JZhao GXu HQiao CHuang H(2025)A cost-efficient traffic engineering framework with various pricing schemes in cloudsComputer Networks10.1016/j.comnet.2025.111090259(111090)Online publication date: Mar-2025
https://doi.org/10.1016/j.comnet.2025.111090
Cao JGuan YQian KGao JXiao WDong JFu BCai DZhai ESekar VYu MSeneviratne AVeitch D(2024)Crux: GPU-Efficient Communication Scheduling for Deep Learning TrainingProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672239(1-15)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672239
Fatemipour BZhang ZSt-Hilaire M(2024)A Survey on Replica Transfer Optimization Schemes in Geographically Distributed Data CentersIEEE Transactions on Network and Service Management10.1109/TNSM.2024.343716521:6(6301-6317)Online publication date: Dec-2024
https://doi.org/10.1109/TNSM.2024.3437165
Qiu JSong JGuérin RSariowan H(2024)On the Benefits of Traffic “Reprofiling” the Multiple Hops Case—Part IIEEE/ACM Transactions on Networking10.1109/TNET.2024.339203032:4(3421-3436)Online publication date: Aug-2024
https://doi.org/10.1109/TNET.2024.3392030
Song JQiu JGuérin RSariowan H(2024)On the Benefits of Traffic “Reprofiling” the Single Hop CaseIEEE/ACM Transactions on Networking10.1109/TNET.2024.335686332:3(2511-2524)Online publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1109/TNET.2024.3356863
Tang SYu YWang HWang GChen WXu ZGuo SGao W(2024)A Survey on Scheduling Techniques in Computing and Network ConvergenceIEEE Communications Surveys & Tutorials10.1109/COMST.2023.332902726:1(160-195)Online publication date: Sep-2025
https://doi.org/10.1109/COMST.2023.3329027
Zhang HYin XShi XWang JWang ZGuo YLan TLi YZhu YRuan KGeng H(2023)Achieving High Availability in Inter-DC WAN Traffic EngineeringIEEE/ACM Transactions on Networking10.1109/TNET.2022.321659231:6(2406-2421)Online publication date: Dec-2023
https://doi.org/10.1109/TNET.2022.3216592
Wang SHou AWu CQu X(2023)On Periodic Point-to-Point Bandwidth Scheduling in High-Performance Networks2023 IEEE International Conference on High Performance Computing & Communications, Data Science & Systems, Smart City & Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys)10.1109/HPCC-DSS-SmartCity-DependSys60770.2023.00078(528-535)Online publication date: 17-Dec-2023
https://doi.org/10.1109/HPCC-DSS-SmartCity-DependSys60770.2023.00078
Yang ZCui YWang XLi MLiu Y(2022)Less is More: Service Profit Maximization in Geo-Distributed CloudsIEEE Transactions on Cloud Computing10.1109/TCC.2020.302461610:3(1925-1940)Online publication date: 1-Jul-2022
https://doi.org/10.1109/TCC.2020.3024616
Tao XOta KDong MBorjigin WQi HLi K(2022)Congestion-Aware Traffic Allocation for Geo-Distributed Data CentersIEEE Transactions on Cloud Computing10.1109/TCC.2020.300119210:3(1675-1687)Online publication date: 1-Jul-2022
https://doi.org/10.1109/TCC.2020.3001192
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten