More Web Proxy on the site http://driver.im/

research-article

Public Access

The Slow Path Needs an Accelerator Too!

Authors:

Annus Zulfiqar,

Gianni Antichi,

Muhammad ShahbazAuthors Info & Claims

ACM SIGCOMM Computer Communication Review, Volume 53, Issue 1

Pages 38 - 47

https://doi.org/10.1145/3594255.3594259

Published: 20 April 2023 Publication History

Abstract

Packet-processing data planes have been continuously enhanced in performance over the last few years to the point that, nowadays, they are increasingly implemented in hardware (i.e., in SmartNICs and programmable switches). However, little attention is given to the slow path residing between the data plane and the control plane, as it is not typically considered performance-critical.

In this paper, we show that the slow path is set to become a new key bottleneck in Software-Defined Networks (SDNs). This is due to the growth in physical network bandwidth (200 Gbps is becoming common in data centers) and topological complexity (e.g., virtual switches now span hundreds of physical machines). We present our vision of a new Domain Specific Accelerator (DSA) for the slow path at the end host that sits between the hardware-offloaded data plane and the logically-centralized control plane. We discuss open problems in this domain and call on the networking community to creatively address this emerging issue.

References

[1]

AMD. last accessed: 11/28/2022. Pensando. https://www.amd.com/en/accelerators/pensando.

[2]

AMD. last accessed: 11/28/2022. Pensando DSC-200 Distributed Services Card. https://www.amd.com/system/files/documents/pensando-dsc-200-product-brief.pdf.

[3]

Gianni Antichi and Gábor Rétvári. 2020. Full-Stack SDN: The Next Big Challenge?. In SOSR.

[4]

Antrea last accessed: 11/28/2022. Antrea: Enhance pod networking and enforce network policies for Kubernetes clusters. https://antrea.io/.

[5]

Abhik Bose, Shailendra Kirtikar, Shivaji Chirumamilla, Rinku Shah, and Mythili Vutukuru. 2022. AccelUPF: Accelerating the 5G User Plane Using Programmable Hardware. In SOSR.

[6]

Pat Bosshart, Glen Gibb, Hun-Seok Kim, George Varghese, Nick McKeown, Martin Izzard, Fernando Mujica, and Mark Horowitz. 2013. Forwarding Metamorphosis: Fast Programmable Match-Action Processing in Hardware for SDN. In ACM SIGCOMM.

Digital Library

[7]

BROADCOM. last accessed: 11/28/2022. BCM88690. https://www.broadcom.com/products/ethernet-connectivity/switching/stratadnx/bcm88690.

[8]

BROADCOM. last accessed: 11/28/2022. Trident4 / BCM56880 Series. https://www.broadcom.com/products/ethernet-connectivity/switching/strataxgs/bcm56880-series.

[9]

CAIDA. last accessed: 11/30/2022. The CAIDA UCSD anonymized internet traces. https://www.caida.org/catalog/datasets/passive_dataset/.

[10]

Bradley Cain, Dr. Steve E. Deering, Bill Fenner, Isidor Kouvelas, and Ajit Thyagarajan. last accessed: 11/30/2022. Internet Group Management Protocol, Version 3. https://www.rfc-editor.org/info/rfc3376.

[11]

Levente Csikor, Dinil Mon Divakaran, Min Suk Kang, Attila Kőrösi, Balázs Sonkoly, Dávid Haja, Dimitrios P. Pezaros, Stefan Schmid, and Gábor Rétvári. 2019. Tuple Space Explosion: A Denial-of-Service Attack against a Software Packet Classifier. In ACM CoNEXT.

[12]

Alexis de Talhouët. last accessed: 11/30/2022. The evolution of SDN: What service mesh offers telco. https://www.redhat.com/en/blog/evolution-sdn-what-service-mesh-offers-telco.

[13]

Daniel Firestone. 2017. VFP: A Virtual Switch Platform for Host SDN in the Public Cloud. In USENIX NSDI.

[14]

Daniel Firestone, Andrew Putnam, Sambhrama Mundkur, Derek Chiou, Alireza Dabagh, Mike Andrewartha, Hari Angepat, Vivek Bhanu, Adrian Caulfield, Eric Chung, Harish Kumar Chandrappa, Somesh Chaturmohta, Matt Humphrey, Jack Lavier, Norman Lam, Fengfen Liu, Kalin Ovtcharov, Jitu Padhye, Gautham Popuri, Shachar Raindel, Tejas Sapre, Mark Shaw, Gabriel Silva, Madhan Sivakumar, Nisheeth Srivastava, Anshuman Verma, Qasim Zuhair, Deepak Bansal, Doug Burger, Kushagra Vaid, David A. Maltz, and Albert Greenberg. 2018. Azure Accelerated Networking: SmartNICs in the Public Cloud. In USENIX NSDI.

[15]

Open Networking Foundation. last accessed: 11/28/2022. Stratum - Enabling the Era of Next Generation SDN. https://opennetworking.org/stratum/.

[16]

Malvika Gupta. last accessed: 11/30/2022. Open vSwitch Offload by SmartNICs on Arm. https://community.arm.com/arm-community-blogs/b/tools-software-ides-blog/posts/open-vswitch-offload-by-smartnics-on-arm.

[17]

Thomas Holterbach, Edgar Costa Molero, Maria Apostolaki, Alberto Dainotti, Stefano Vissicchio, and Laurent Vanbever. 2019. Blink: Fast Connectivity Recovery Entirely in the Data Plane. In USENIX NSDI.

[18]

Serve The Home. last accessed: 11/28/2022. Intel X710 OCP NIC 3.0 Power Consumption Specs. https://www.servethehome.com/intel-x710-da2-ocp-nic-3-0-review-10gbe-for-the-form-factor/intel-x710-ocp-nic-3-0-power-consumption-specs/.

[19]

Serve The Home. last accessed: 11/28/2022. Pensando Distributed Services Architecture SmartNIC. https://www.servethehome.com/pensando-distributed-services-architecture-smartnic/.

[20]

IEEE. last accessed: 11/30/2022. IEEE Standard for Information Technology - Local and Metropolitan Area Networks - Part 3: Carrier Sense Multiple Access with Collision Detection (CSMA/CD) Access Method and Physical Layer Specifications-Aggregation of Multiple Link Segments. https://standards.ieee.org/ieee/802.3ad/1088/.

[21]

IEEE. last accessed: 11/30/2022. IEEE Standard for Local and Metropolitan Area Networks Virtual Bridged Local Area Networks Amendment 5: Connectivity Fault Management. http://standards.ieee.org/getieee802/download/802.1ag-2007.pdf.

[22]

Intel. last accessed: 11/30/2022. Intel Ethernet Controller 700 Series - Open vSwitch Hardware Acceleration Application Note. https://builders.intel.com/docs/networkbuilders/intel-ethernet-controller-700-series-open-vswitch-hardware-acceleration-application-note.pdf.

[23]

Intel. last accessed: 11/30/2022. Intel Infrastructure Processing Units (IPUs) and Smart-NICs. https://www.intel.com/content/www/us/en/products/details/network-io/ipu.html.

[24]

Intel. last accessed: 11/30/2022. Tofino: P4-programmable Ethernet switch ASIC that delivers better performance at lower power. https://www.intel.com/content/www/us/en/products/network-io/programmable-ethernet-switch/tofino-series.html.

[25]

Intel. last accessed: 11/30/2022. Tofino2: Second-generation P4-programmable Ethernet Switch ASIC that Continues to Deliver Programmability without Compromise. https://www.intel.com/content/www/us/en/products/network-io/programmable-ethernet-switch/tofino-2-series.html.

[26]

Istio. last accessed: 11/30/2022. Simplify observability, traffic management, security, and policy with the leading service mesh. https://istio.io/.

[27]

Anjali Singhai Jain, Mrittika Ganguli, Valas Valancius, and Nupur Jain. last accessed: 11/30/2022. Service Mesh P4 Data Plane. https://opennetworking.org/2022-p4-workshop-gated/.

[28]

Sushant Jain, Alok Kumar, Subhasree Mandal, Joon Ong, Leon Poutievski, Arjun Singh, Subbaiah Venkata, Jim Wanderer, Junlan Zhou, Min Zhu, Jon Zolla, Urs Hölzle, Stephen Stuart, and Amin Vahdat. 2013. B4: Experience with a Globally-Deployed Software Defined WAN. In ACM SIGCOMM.

[29]

Georgios Katsikas, Tom Barbette, Marco Chiesa, Dejan Kostic, and Gerald Maguire. 2021. What You Need to Know About (Smart) Network Interface Cards. In International Conference on Passive and Active Network Measurement (PAM).

Digital Library

[30]

Dave Katz and David Ward. last accessed: 11/30/2022. Bidirectional Forwarding Detection (BFD). https://www.rfc-editor.org/info/rfc5880.

[31]

Kinvolk.io. last accessed: 11/30/2022. Performance Benchmarks Analysis of Istio and Linkerd. https://kinvolk.io/blog/2019/05/performance-benchmark-analysis-of-istio-and-linkerd.

[32]

Patricia Kummrow. last accessed: 11/30/2022. The IPU: A New, Strategic Resource for Cloud Service Providers. https://community.intel.com/t5/Blogs/Tech-Innovation/Data-Center/The-IPU-A-New-Strategic-Resource-for-Cloud-Service-Providers/post/1335081.

[33]

Robert MacDavid, Carmelo Cascone, Pingping Lin, Badhrinath Padmanabhan, Ajay ThakuR, Larry Peterson, Jennifer Rexford, and Oguz Sunay. 2021. A P4-Based 5G User Plane Function. In SOSR.

[34]

Michael Marty, Marc de Kruijf, Jacob Adriaens, Christopher Alfeld, Sean Bauer, Carlo Contavalli, Michael Dalton, Nandita Dukkipati, William C. Evans, Steve Gribble, Nicholas Kidd, Roman Kononov, Gautam Kumar, Carl Mauer, Emily Musick, Lena Olson, Erik Rubow, Michael Ryan, Kevin Springborn, Paul Turner, Valas Valancius, Xi Wang, and Amin Vahdat. 2019. Snap: A Microkernel Approach to Host Networking. In SOSP.

[35]

Nick McKeown, Tom Anderson, Hari Balakrishnan, Guru Parulkar, Larry Peterson, Jennifer Rexford, Scott Shenker, and Jonathan Turner. 2008. OpenFlow: Enabling Innovation in Campus Networks. ACM SIGCOMM CCR (2008).

Digital Library

[36]

Edgar Costa Molero, Stefano Vissicchio, and Laurent Vanbever. 2018. Hardware-Accelerated Network Control Planes. In HotNets.

[37]

Napatech. last accessed: 11/30/2022. UPF offload for Napatech Programmable SmartNICs. https://www.napatech.com/support/resources/data-sheets/5g-user-plane-function-upf-offload/.

[38]

Arvind Narayanan, Xumiao Zhang, Ruiyang Zhu, Ahmad Hassan, Shuowei Jin, Xiao Zhu, Xiaoxuan Zhang, Denis Rybkin, Zhengxuan Yang, Zhuoqing Morley Mao, Feng Qian, and Zhi-Li Zhang. 2021. A Variegated Look at 5G in the Wild: Performance, Power, and QoE Implications. In ACM SIGCOMM.

[39]

Nvidia. last accessed: 11/30/2022. CONNECTX-6 DX. https://www.nvidia.com/en-us/networking/ethernet/connectx-6-dx/.

[40]

Nvidia. last accessed: 11/30/2022. NVIDIA BLUEFIELD DATA PROCESSING UNITS. https://www.nvidia.com/en-us/networking/products/data-processing-unit/.

[41]

NVIDIA. last accessed: 11/30/2022. OVS Offload Using ASAP2 Direct. https://docs.nvidia.com/networking/display/MLNXENv531001/OVS+Offload+Using+ASAP2+Direct.

[42]

ONOS last accessed: 11/30/2022. ONOS: Open Network Operating System. https://opennetworking.org/onos/.

[43]

Open Daylight last accessed: 11/30/2022. Open Daylight: modular open platform for customizing and automating networks of any size and scale. https://www.opendaylight.org/.

[44]

Open-vSwitch. last accessed: 11/30/2022. Open vSwitch with DPDK. https://docs.openvswitch.org/en/latest/intro/install/dpdk/.

[45]

Open-vSwitch. last accessed: 11/30/2022. ovs-vswitchd(8). https://www.openvswitch.org/support/dist-docs-2.5/ovs-vswitchd.8.txt.

[46]

Github Open-vSwitch. last accessed: 11/30/2022. ofproto-dpif-upcall.c. https://github.com/openvswitch/ovs/blob/master/ofproto/ofproto-dpif-upcall.c.

[47]

Francesco Paolucci, Davide Scano, Filippo Cugini, Andrea Sgambelluri, Luca Valcarenghi, Carlo Cavazzoni, Giuseppe Ferraris, and Piero Castoldi. 2021. User Plane Function Offloading in P4 switches for enhanced 5G Mobile Edge Computing. In International Conference on the Design of Reliable Communication Networks (DRCN).

[48]

Larry L. Peterson, Carmelo Cascone, Brian O'Connor, Thomas Vachuska, and Bruce Davie. 2021. Software-Defined Networks: A Systems Approach. Systems Approach LLC.

[49]

Larry L. Peterson and Bruce S. Davie. 2021. Computer Networks: A Systems Approach (9th ed.). Morgan Kaufmann.

[50]

Ben Pfaff and Bruce Davie. last accessed: 11/30/2022. The Open vSwitch Database Management Protocol. https://www.rfc-editor.org/info/rfc7047.

[51]

Ben Pfaff and Jesse Gross. last accessed: 11/30/2022. Open vSwitch datapath developer documentation. https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Documentation/networking/openvswitch.rst.

[52]

Ben Pfaff, Justin Pettit, Teemu Koponen, Ethan Jackson, Andy Zhou, Jarno Rajahalme, Jesse Gross, Alex Wang, Joe Stringer, Pravin Shelar, Keith Amidon, and Martin Casado. 2015. The Design and Implementation of Open vSwitch. In USENIX NSDI.

[53]

Diana Andreea Popescu. last accessed: 11/30/2022. Latency-driven performance in data centers. https://www.cl.cam.ac.uk/techreports/UCAM-CL-TR-937.pdf.

[54]

Alon Rashelbach, Ori Rottenstreich, and Mark Silberstein. 2020. A Computational Approach to Packet Classification. In ACM SIGCOMM.

[55]

Alon Rashelbach, Ori Rottenstreich, and Mark Silberstein. 2022. Scaling Open vSwitch with a Computational Cache. In USENIX NSDI.

[56]

Redhat. last accessed: 11/30/2022. What's a service mesh? https://www.redhat.com/en/topics/microservices/what-is-a-service-mesh.

[57]

Gerald Rogers and Pravin Shelar. last accessed: 11/30/2022. Using Open vSwitch with DPDK. https://github.com/openvswitch/ovs/blob/master/Documentation/howto/dpdk.rst.

[58]

Richard Sanger, Brad Cowie, Matthew Luckie, and Richard Nelson. 2018. Characterising the Limits of the OpenFlow Slow-Path. In IEEE Conference on Network Function Visualization and Software Defined Networks (NFV-SDN).

[59]

Muhammad Shahbaz, Lalith Suresh, Jennifer Rexford, Nick Feamster, Ori Rottenstreich, and Mukesh Hira. 2019. Elmo: Source Routed Multi-cast for Public Clouds. In ACM SIGCOMM.

Digital Library

[60]

Arjun Singhvi, Aditya Akella, Dan Gibson, Thomas F. Wenisch, Monica Wong-Chan, Sean Clark, Milo M. K. Martin, Moray McLaren, Prashant Chandra, Rob Cauble, Hassan M. G. Wassel, Behnam Montazeri, Simon L. Sabato, Joel Scherpelz, and Amin Vahdat. 2020. 1RMA: Re-Envisioning Remote Memory Access for Multi-Tenant Datacenters. In ACM SIGCOMM.

[61]

Anirudh Sivaraman, Suvinay Subramanian, Mohammad Alizadeh, Sharad Chole, Shang-Tse Chuang, Anurag Agrawal, Hari Balakrishnan, Tom Edsall, Sachin Katti, and Nick McKeown. 2016. Programmable Packet Scheduling at Line Rate. In ACM SIGCOMM.

[62]

Tushar Swamy, Alexander Rucker, Muhammad Shahbaz, Ishan Gaur, and Kunle Olukotun. 2022. Taurus: A Data Plane Architecture for per-Packet ML. In ASPLOS.

[63]

David E. Taylor and Jonathan S. Turner. 2007. ClassBench: A Packet Classification Benchmark. IEEE/ACM Transations on Networking (2007).

[64]

Tigera. last accessed: 11/30/2022. Project Calico. https://www.tigera.io/project-calico/.

[65]

Jean Tourrilhes, Justin Pettit, et al. last accessed: 11/30/2022. Open-Flow Switch Specification, Version 1.5.1 (Protocol version 0×06). https://opennetworking.org/wp-content/uploads/2014/10/openflow-switch-v1.5.1.pdf.

[66]

William Tu, Yi-Hung Wei, Gianni Antichi, and Ben Pfaff. 2021. Revisiting the Open VSwitch Dataplane Ten Years Later. In ACM SIGCOMM.

[67]

VMware. last accessed: 11/30/2022. VMware NSX: Network Virtualization Platform. https://www.vmware.com/products/nsx.html.

[68]

VMware. last accessed: 11/30/2022. VMware's per-CPU Pricing Model. https://news.vmware.com/company/cpu-pricing-model-update-feb-2020.

[69]

Open vSwitch. last accessed: 11/30/2022. Open vSwitch Manual. http://www.openvswitch.org/support/dist-docs/ovs-vswitchd.conf.db.5.pdf.

[70]

Yong Wang, Boon Ang, Guolin Yang, and Wengyi Jiang. last accessed: 11/30/2022. BFD Offload in Virtual Network Interface Controller. https://patents.google.com/patent/US11196651B2/en?oq=%2316%2f661%2c879.

[71]

Xilinx. last accessed: 11/30/2022. Alveo SN1000 SmartNICs. https://www.xilinx.com/content/dam/xilinx/publications/product-briefs/sn1000-product-brief.pdf.

[72]

Xilinx. last accessed: 11/30/2022. Alveo U25 SmartNIC. https://www.xilinx.com/products/boards-and-kits/alveo/u25.html.

[73]

Xilinx. last accessed: 11/30/2022. OVS Offload. https://www.xilinx.com/publications/solution-briefs/partner/vvdn-ovs-solution-brief.pdf.

[74]

Siyu Yan, Xiaoliang Wang, Xiaolong Zheng, Yinben Xia, Derui Liu, and Weishan Deng. 2021. ACC: Automatic ECN Tuning for High-Speed Datacenter Networks. In ACM SIGCOMM.

Digital Library

[75]

Liangcheng Yu, John Sonchack, and Vincent Liu. 2020. Mantis: Reactive Programmable Switches. In ACM SIGCOMM.

Digital Library

[76]

Lihua Yuan. last accessed: 11/30/2022. SONiC: Software for Open Networking in the Cloud. https://conferences.sigcomm.org/events/apnet2018/slides/lihua.pdf.

Cited By

Chen XLiu HZhang WHuang QZhang DZhou HLiu XWu C(2024)Toward Full-Coverage and Low-Overhead Profiling of Network-Stack LatencyIEEE/ACM Transactions on Networking10.1109/TNET.2024.342132732:5(4441-4455)Online publication date: Oct-2024
https://doi.org/10.1109/TNET.2024.3421327
Saleh SGoossens AShu SBanerjee TKoldehofe B(2024)Analog In-Network Computing through Memristor-based Match-Compute ProcessingIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621228(2518-2527)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621228
Deri LCardigliano AFusco F(2024)Advancements in Traffic Processing Using Programmable Hardware Flow Offload2024 IEEE 25th International Conference on High Performance Switching and Routing (HPSR)10.1109/HPSR62440.2024.10635975(61-66)Online publication date: 22-Jul-2024
https://doi.org/10.1109/HPSR62440.2024.10635975

Index Terms

The Slow Path Needs an Accelerator Too!
1. Hardware
  1. Communication hardware, interfaces and storage
    1. Networking hardware
2. Networks
  1. Network services
    1. In-network processing
    2. Programmable networks

Recommendations

P4: programming protocol-independent packet processors

P4 is a high-level language for programming protocol-independent packet processors. P4 works in conjunction with SDN control protocols like OpenFlow. In its current form, OpenFlow explicitly specifies protocol headers on which it operates. This set has ...
Towards a more programmable and performance-optimized Virtual Switch: Introducing P4 programmability within OVS
ANCS '21: Proceedings of the Symposium on Architectures for Networking and Communications Systems

Advent of network function virtualization and multiple virtual machines hosting multiple applications fueled the need for a programmable software switch that coordinates network traffic across them. Open Virtual Switch (OVS) is a software component that ...
Characterizing the Performance of Concurrent Virtualized Network Functions with OVS-DPDK, FD.IO VPP and SR-IOV
ICPE '18: Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering

The virtualization of network functions is promising significant cost reductions for network operators. Running multiple network functions on a standard x86 server instead of dedicated appliances can increase the utilization of the underlying hardware,...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM SIGCOMM Computer Communication Review

ACM SIGCOMM Computer Communication Review Volume 53, Issue 1

January 2023

70 pages

ISSN:0146-4833

DOI:10.1145/3594255

Editor:
Steve Uhlig
School of Electronic Engineering and Computer Science, Queen Mary, University of London, London, United Kingdom

Issue’s Table of Contents

Copyright © 2023 Copyright is held by the owner/author(s).

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2023

Published in SIGCOMM-CCR Volume 53, Issue 1

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

European Union
NSF
ACE

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
562
Total Downloads

Downloads (Last 12 months)343
Downloads (Last 6 weeks)30

Reflects downloads up to 13 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen XLiu HZhang WHuang QZhang DZhou HLiu XWu C(2024)Toward Full-Coverage and Low-Overhead Profiling of Network-Stack LatencyIEEE/ACM Transactions on Networking10.1109/TNET.2024.342132732:5(4441-4455)Online publication date: Oct-2024
https://doi.org/10.1109/TNET.2024.3421327
Saleh SGoossens AShu SBanerjee TKoldehofe B(2024)Analog In-Network Computing through Memristor-based Match-Compute ProcessingIEEE INFOCOM 2024 - IEEE Conference on Computer Communications10.1109/INFOCOM52122.2024.10621228(2518-2527)Online publication date: 20-May-2024
https://doi.org/10.1109/INFOCOM52122.2024.10621228
Deri LCardigliano AFusco F(2024)Advancements in Traffic Processing Using Programmable Hardware Flow Offload2024 IEEE 25th International Conference on High Performance Switching and Routing (HPSR)10.1109/HPSR62440.2024.10635975(61-66)Online publication date: 22-Jul-2024
https://doi.org/10.1109/HPSR62440.2024.10635975
Kfoury EChoueiri SMazloum AAlSabeh AGomez JCrichigno J(2024)A Comprehensive Survey on SmartNICs: Architectures, Development Models, Applications, and Research DirectionsIEEE Access10.1109/ACCESS.2024.343720312(107297-107336)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3437203

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents