poster

Poster: Chameleon: Automatic and Adaptive Tuning for DCQCN Parameters in RDMA Networks

Authors:

Ziteng Chen,

Menghao Zhang,

Guanyu Li,

Mingwei XuAuthors Info & Claims

ACM SIGCOMM '23: Proceedings of the ACM SIGCOMM 2023 Conference

Pages 1091 - 1093

https://doi.org/10.1145/3603269.3610865

Published: 01 September 2023 Publication History

Get Access

Abstract

Datacenter Quantized Congestion Notification (DCQCN) [12] is the default congestion control algorithm for Mellanox RDMA (Remote Direct Memory Access) NICs [2] in RoCEv2 (RDMA over Converged Ethernet v2) networks, one of the most widely used NICs in leading industry companies [4, 5, 7, 9]. In DCQCN, firstly switches mark packets with ECN (Explicit Congestion Notification) when the queue length exceeds ECN thresholds, then receivers respond to ECN-marked packets with CNPs (Congestion Notification Packets), and finally senders reduce transmission rate when receiving CNPs. DCQCN has 10+ parameters at both NICs and switches, including Alpha Update, Rate Increase & Decrease, Notification Point and ECN thresholds [3], and these parameters have a non-negligible impact on the network performance. Our experiments also verify the network performance of common AI (Artificial Intelligence) training workloads in RoCEv2 networks (e.g., all-to-all collective communication) is greatly influenced by different DCQCN parameter settings (^§3). Therefore, when deploying applications in practice, the DCQCN parameters need to be carefully tested and tuned to improve the network performance.

References

[1]

2020. High Precision Congestion Control. (2020). https://github.com/alibaba-edu/High-Precision-Congestion-Control

Google Scholar

[2]

2022. DCQCN CC Algorithm. (2022). https://enterprise-support.nvidia.com/s/article/DCQCN-CC-algorithm

Google Scholar

[3]

2023. DCQCN Parameters. (2023). https://enterprise-support.nvidia.com/s/article/dcqcn-parameters

Google Scholar

[4]

Wei Bai, Shanim Sainul Abdeen, Ankit Agrawal, Krishan Kumar Attre, Paramvir Bahl, Ameya Bhagat, Gowri Bhaskara, Tanya Brokhman, Lei Cao, Ahmad Cheema, et al. 2023. Empowering Azure Storage with RDMA. In NSDI.

Google Scholar

[5]

Yixiao Gao, Qiang Li, Lingbo Tang, Yongqing Xi, Pengcheng Zhang, Wenwen Peng, Bo Li, Yaohui Wu, Shaozong Liu, Lei Yan, et al. 2021. When Cloud Storage Meets RDMA. In NSDI.

Google Scholar

[6]

Yixiao Gao, Yuchen Yang, Tian Chen, Jiaqi Zheng, Bing Mao, and Guihai Chen. 2018. Dcqcn+: Taming large-scale incast congestion in rdma over ethernet networks. In ICNP.

Google Scholar

[7]

Yimin Jiang, Yibo Zhu, Chang Lan, Bairen Yi, Yong Cui, and Chuanxiong Guo. 2020. A unified architecture for accelerating distributed DNN training in heterogeneous GPU/CPU clusters. In OSDI.

Google Scholar

[8]

Yuliang Li, Rui Miao, Hongqiang Harry Liu, Yan Zhuang, Fei Feng, Lingbo Tang, Zheng Cao, Ming Zhang, Frank Kelly, Mohammad Alizadeh, et al. 2019. HPCC: High precision congestion control. In SIGCOMM.

Google Scholar

[9]

Dheevatsa Mudigere, Yuchen Hao, Jianyu Huang, Zhihao Jia, Andrew Tulloch, Srinivas Sridharan, Xing Liu, Mustafa Ozdal, Jade Nie, Jongsoo Park, et al. 2022. Software-hardware co-design for fast and scalable training of deep learning recommendation models. In ISCA.

Google Scholar

[10]

Kai Wang, Fang Dong, Dian Shen, Chengtian Zhang, Jinghui Zhang, and Junzhou Luo. 2021. Towards tunable RDMA parameter selection at runtime for datacenter applications. In CSCWD.

Google Scholar

[11]

Siyu Yan, Xiaoliang Wang, Xiaolong Zheng, Yinben Xia, Derui Liu, and Weishan Deng. 2021. ACC: Automatic ECN tuning for high-speed datacenter networks. In SIGCOMM.

Digital Library

Google Scholar

[12]

Yibo Zhu, Haggai Eran, Daniel Firestone, Chuanxiong Guo, Marina Lipshteyn, Yehonatan Liron, Jitendra Padhye, Shachar Raindel, Mohamad Haj Yahia, and Ming Zhang. 2015. Congestion control for large-scale RDMA deployments. In SIGCOMM.

Google Scholar

Cited By

View all

Luangsomboon NLiebeherr J(2024)Necessary and Sufficient Condition for Triggering ECN Before PFC in Shared Memory SwitchesIEEE Networking Letters10.1109/LNET.2024.33829556:2(119-123)Online publication date: Jun-2024
https://doi.org/10.1109/LNET.2024.3382955
Zhang YFan YXiao YChen HXie PLiu LMa H(2024)BBQ: Dynamic-Buffer-Driven Automatic ECN Tunning in Datacenter2024 IEEE/ACM 32nd International Symposium on Quality of Service (IWQoS)10.1109/IWQoS61813.2024.10682919(1-6)Online publication date: 19-Jun-2024
https://doi.org/10.1109/IWQoS61813.2024.10682919
Merino AEscudero-Sahuquillo JGarcia PQuiles FChen FLyu YYan LDuato J(2024)A Hybrid Solution to Provide End-to-End Flow Control and Congestion Management in High-Performance Interconnection Networks2024 IEEE 24th International Symposium on Cluster, Cloud and Internet Computing (CCGrid)10.1109/CCGrid59990.2024.00011(8-17)Online publication date: 6-May-2024
https://doi.org/10.1109/CCGrid59990.2024.00011

Index Terms

Poster: Chameleon: Automatic and Adaptive Tuning for DCQCN Parameters in RDMA Networks
1. Networks
  1. Network protocols
    1. Transport protocols

Recommendations

Congestion Control for Large-Scale RDMA Deployments
SIGCOMM '15: Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication

Modern datacenter applications demand high throughput (40Gbps) and ultra-low latency (< 10 μs per hop) from the network, with low CPU overhead. Standard TCP/IP stacks cannot meet these requirements, but Remote Direct Memory Access (RDMA) can. On IP-...
Congestion Control for Large-Scale RDMA Deployments
SIGCOMM'15

Modern datacenter applications demand high throughput (40Gbps) and ultra-low latency (< 10 μs per hop) from the network, with low CPU overhead. Standard TCP/IP stacks cannot meet these requirements, but Remote Direct Memory Access (RDMA) can. On IP-...
Unreliable transport protocol using congestion control for high-speed networks

Currently there is no control for the real-time traffic of multimedia applications using UDP (User Datagram Protocol) in high-speed networks. Therefore, although a number of high-speed TCP (Transmission Control Protocol) protocols have been developed ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ACM SIGCOMM '23: Proceedings of the ACM SIGCOMM 2023 Conference

September 2023

1217 pages

ISBN:9798400702365

DOI:10.1145/3603269

Chairs:
Henning Schulzrinne,
Vishal Misra,
Program Chairs:
Eddie Kohler,
David Maltz

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 September 2023

Check for updates

Author Tags

Qualifiers

Poster

Funding Sources

National Natural Science Foundation of China
China Postdoctoral Science Foundation
Beijing Postdoctoral Research Foundation

Conference

ACM SIGCOMM '23

Sponsor:

SIGCOMM

ACM SIGCOMM '23: ACM SIGCOMM 2023 Conference

September 10, 2023

NY, New York, USA

Acceptance Rates

Overall Acceptance Rate 462 of 3,389 submissions, 14%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
350
Total Downloads

Downloads (Last 12 months)233
Downloads (Last 6 weeks)20

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Luangsomboon NLiebeherr J(2024)Necessary and Sufficient Condition for Triggering ECN Before PFC in Shared Memory SwitchesIEEE Networking Letters10.1109/LNET.2024.33829556:2(119-123)Online publication date: Jun-2024
https://doi.org/10.1109/LNET.2024.3382955
Zhang YFan YXiao YChen HXie PLiu LMa H(2024)BBQ: Dynamic-Buffer-Driven Automatic ECN Tunning in Datacenter2024 IEEE/ACM 32nd International Symposium on Quality of Service (IWQoS)10.1109/IWQoS61813.2024.10682919(1-6)Online publication date: 19-Jun-2024
https://doi.org/10.1109/IWQoS61813.2024.10682919
Merino AEscudero-Sahuquillo JGarcia PQuiles FChen FLyu YYan LDuato J(2024)A Hybrid Solution to Provide End-to-End Flow Control and Congestion Management in High-Performance Interconnection Networks2024 IEEE 24th International Symposium on Cluster, Cloud and Internet Computing (CCGrid)10.1109/CCGrid59990.2024.00011(8-17)Online publication date: 6-May-2024
https://doi.org/10.1109/CCGrid59990.2024.00011

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Congestion Control for Large-Scale RDMA Deployments

Congestion Control for Large-Scale RDMA Deployments

Unreliable transport protocol using congestion control for high-speed networks