More Web Proxy on the site http://driver.im/

research-article

Open access

Particle: ephemeral endpoints for serverless networking

Authors:

Geoffrey M. Voelker,

George PorterAuthors Info & Claims

SoCC '20: Proceedings of the 11th ACM Symposium on Cloud Computing

Pages 16 - 29

https://doi.org/10.1145/3419111.3421275

Published: 12 October 2020 Publication History

Abstract

Burst-parallel serverless applications invoke thousands of short-lived distributed functions to complete complex jobs such as data analytics, video encoding, or compilation. While these tasks execute in seconds, starting and configuring the virtual network they rely on is a major bottleneck that can consume up to 84% of total startup time. In this paper we characterize the magnitude of this network cold start problem in three popular overlay networks, Docker Swarm, Weave, and Linux Overlay. We focus on end-to-end startup time that encompasses both the time to boot a group of containers as well as interconnecting them. Our primary observation is that existing overlay approaches for serverless networking scale poorly in short-lived serverless environments. Based on our findings we develop Particle, a network stack tailored for multi-node serverless overlay networks that optimizes network creation without sacrificing multi-tenancy, generality, or throughput. When integrated into a serverless burst-parallel video processing pipeline, Particle improves application runtime by 2.4--3X over existing overlays.

Supplementary Material

MP4 File (p16-thomas-presentation.mp4)

Download
13.63 MB

References

[1]

Alexandra Agache, Marc Brooker, Andreea Florescu, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa. Firecracker: Lightweight Virtualization for Serverless Applications. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20), Santa Clara, CA, February 2020. USENIX Association.

[2]

Istemi Ekin Akkus, Ruichuan Chen, Ivica Rimac, Manuel Stein, Klaus Satzke, Andre Beck, Paarijaat Aditya, and Volker Hilt. SAND: Towards High-performance Serverless Computing. In Proceedings of the 2018 USENIX Annual Technical Conference, USENIX ATC'18, pages 923--935, Berkeley, CA, USA, 2018. USENIX Association.

[3]

Amazon Web Services. Amazon Elastic Container Service. https://aws.amazon.com/ecs/, 2020.

[4]

Lixiang Ao, Liz Izhikevich, Geoffrey M. Voelker, and George Porter. Sprocket: A Serverless Video Processing Framework. In Proceedings of the ACM Symposium on Cloud Computing, SoCC'18, pages 263--274. ACM, 2018.

[5]

ARM. Quagga Routing Suite. https://www.nongnu.org/quagga/, 2018.

[6]

Calico. Calico. https://www.projectcalico.org/, 2019.

[7]

CoreOS. Flannel. https://coreos.com/flannel, 2019.

[8]

Jeffrey Dean and Sanjay Ghemawat. MapReduce: Simplified Data Processing on Large Clusters. Communications of the ACM, 51(1):107--113, January 2008.

Digital Library

[9]

Docker. Docker. https://www.docker.com/, 2017.

[10]

Sadjad Fouladi, Francisco Romero, Dan Iter, Qian Li, Shuvo Chatterjee, Christos Kozyrakis, Matei Zaharia, and Keith Winstein. From Laptop to Lambda: Outsourcing Everyday Jobs to Thousands of Transient Functional Containers. In Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC'19, pages 475--488, 2019.

[11]

Sadjad Fouladi, Riad S Wahby, Brennan Shacklett, Karthikeyan Vasuki Balasubramaniam, William Zeng, Rahul Bhalerao, Anirudh Sivaraman, George Porter, and Keith Winstein. Encoding, Fast and Slow: Low-Latency Video Processing using Thousands of Tiny Threads. In Proceedings of the 14th USENIX Symposium on Networked Systems Design and Implementation, NSDI'17, pages 363--376, 2017.

[12]

Richard L Graham, Timothy S Woodall, and Jeffrey M Squyres. Open MPI: A Flexible High Performance MPI. In Proceedings of the 6th International Conference on Parallel Processing and Applied Mathematics, PPAM'05, pages 228--239. Springer, 2005.

[13]

IBM. OpenWhisk Github. https://github.com/apache/openwhisk, 2020.

[14]

iproute2. Iproute2 routing commands. https://git.kernel.org/pub/scm/network/iproute2/iproute2.git, 2019.

[15]

Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, and Dennis Fetterly. Dryad: Distributed Data-parallel Programs from Sequential Building Blocks. In Proceedings of the 2nd ACM European Conference on Computer Systems, EuroSys 07, pages 59--72, Lisbon, Portugal, 2007. ACM.

[16]

Jake Edge. A seccomp overview. https://lwn.net/Articles/656307/, 2015.

[17]

Eric Jonas, Qifan Pu, Shivaram Venkataraman, Ion Stoica, and Benjamin Recht. Occupy the Cloud: Distributed Computing for the 99%. In Proceedings of the 2017 Symposium on Cloud Computing, SoCC'17, pages 445--451, New York, NY, USA, 2017. ACM.

Digital Library

[18]

Eric Jonas, Johann Schleier-Smith, Vikram Sreekanti, Chia-Che Tsai, Anurag Khandelwal, Qifan Pu, Vaishaal Shankar, Joao Carreira, Karl Krauth, Neeraja Yadwadkar, et al. Cloud Programming Simplified: A Berkeley View on Serverless Computing. arXiv preprint arXiv:1902.03383, 2019.

[19]

Kata Containers. Kata Containers. https://katacontainers.io/, 2020.

[20]

Ana Klimovic, Yawen Wang, Patrick Stuedi, Animesh Trivedi, Jonas Pfefferle, and Christos Kozyrakis. Pocket: Elastic Ephemeral Storage for Serverless Analytics. In Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation, OSDI 18, pages 427--444, 2018.

[21]

Ian Lewis. The Almighty Pause Container. https://www.ianlewis.org/en/almighty-pause-container, October 2017.

[22]

Matt Fleming. A thorough introduction to eBPF. https://lwn.net/Articles/740157/, 2017.

[23]

Anup Mohan, Harshad Sane, Kshitij Doshi, Saikrishna Edupuganti, Naren Nayak, and Vadim Sukhomlinov. Agile Cold Starts for Scalable Serverless. In Proceedings of the 11th USENIX Workshop on Hot Topics in Cloud Computing, HotCloud'19, 2019.

[24]

Derek G. Murray, Frank McSherry, Rebecca Isaacs, Michael Isard, Paul Barham, and Martín Abadi. Naiad: A Timely Dataflow System. In Proceedings of the Twenty-Fourth ACM Symposium on Operating Systems Principles, SOSP'13, pages 439--455, New York, NY, USA, 2013. ACM.

[25]

Official Kubernetes. Pod Overview. https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/, 2020.

[26]

Diego Ongaro and John Ousterhout. In Search of an Understandable Consensus Algorithm. In Proceedings of the 2014 USENIX Annual Technical Conference, USENIX ATC'14, pages 305--320, Berkeley, CA, USA, 2014. USENIX Association.

Digital Library

[27]

Qifan Pu, Shivaram Venkataraman, and Ion Stoica. Shuffling, Fast and Slow: Scalable Analytics on Serverless Infrastructure. In Proceedings of the 16th USENIX Symposium on Networked Systems Design and Implementation, NSDI 19, pages 193--206, 2019.

[28]

Alexander Rasmussen, George Porter, Michael Conley, Harsha V. Madhyastha, Radhika Niranjan Mysore, Alexander Pucher, and Amin Vahdat. TritonSort: A Balanced Large-scale Sorting System. In Proceedings of the 8th USENIX Conference on Networked Systems Design and Implementation, NSDI'11, pages 29--42, Berkeley, CA, USA, 2011. USENIX Association.

[29]

Josep Sampé, Gil Vernik, Marc Sánchez-Artigas, and Pedro García-López. Serverless Data Analytics in the IBM Cloud. In Proceedings of the 19th International Middleware Conference Industry, Middleware '18, pages 1--8, New York, NY, USA, 2018. ACM.

[30]

Apache Spark. http://spark.apache.org/.

[31]

Tim Wagner. Serverless Networking is the next step in the evolution of serverless. https://bit.ly/30kFoY9, 2019.

[32]

Liang Wang, Mengyuan Li, Yinqian Zhang, Thomas Ristenpart, and Michael Swift. Peeking Behind the Curtains of Serverless Platforms. In Proceedings of the 2018 USENIX Annual Technical Conference, USENIX ATC'18, pages 133--145, Boston, MA, July 2018. USENIX Association.

[33]

Simple, resilient multi-host containers networking and more. https://github.com/weaveworks/weave.

[34]

Tian Zhang, Dong Xie, Feifei Li, and Ryan Stutsman. Narrowing the gap between serverless and its state with storage functions. In Proceedings of the ACM Symposium on Cloud Computing, SoCC '19, page 1--12, New York, NY, USA, 2019. Association for Computing Machinery.

Digital Library

Cited By

Cvetković LCosta FDjokic MFriedman MKlimovic AWitchel EArpaci-Dusseau ARossbach CKeeton K(2024)Dirigent: Lightweight Serverless OrchestrationProceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles10.1145/3694715.3695966(369-384)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3694715.3695966
Szekely ABelay AMorris RKaashoek MWitchel EArpaci-Dusseau ARossbach CKeeton K(2024)Unifying serverless and microservice workloads with SigmaOSProceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles10.1145/3694715.3695947(385-402)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3694715.3695947
Liu YGuo JJiang BZhang PSun XSong YRen WHou ZLyu BWen RZhu SWang XVallina-Rodríguez NSuarez-Tángil GLevin DPelsser C(2024)Understanding Network Startup for Secure Containers in Multi-Tenant Clouds: Performance, Bottleneck and OptimizationProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688436(635-650)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3646547.3688436
Show More Cited By

Index Terms

Particle: ephemeral endpoints for serverless networking
1. Computer systems organization
  1. Architectures
    1. Distributed architectures
      1. Cloud computing

Recommendations

Dynamic latency control of serverless applications operated on AWS lambda and greengrass
SIGCOMM '20: Proceedings of the SIGCOMM '20 Poster and Demo Sessions

Cloud native programming and the serverless paradigm can revolutionize software development and the operation of distributed applications. However, latency sensitive applications pose additional challenges to the underlying networks and cloud platforms. ...
GridDL: an HTTP bandwidth sharing framework
U-NET '09: Proceedings of the 1st ACM workshop on User-provided networking: challenges and opportunities

Peer-to-peer (P2P) applications have become a mainstream technology for content distribution. Yet widely used P2P applications, such as BitTorrent and Gnutella, suffer from flaws that are currently open topics of research---from the problem of ...
Performance and Cost Comparison of Cloud Services for Deep Learning Workload
ICPE '21: Companion of the ACM/SPEC International Conference on Performance Engineering

Many organizations are migrating their on-premise artificial intelligence workloads to the cloud due to the availability of cost-effective and highly scalable infrastructure, software and platform services. To ease the process of migration, many cloud ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SoCC '20: Proceedings of the 11th ACM Symposium on Cloud Computing

October 2020

535 pages

ISBN:9781450381376

DOI:10.1145/3419111

General Chair:
Rodrigo Fonseca
Microsoft and Brown University
,
Program Chairs:
Christina Delimitrou
Cornell University
,
Beng Chin Ooi
National University of Singapore

Copyright © 2020 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 October 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

NSF

Conference

SoCC '20

Sponsor:

SoCC '20: ACM Symposium on Cloud Computing

October 19 - 21, 2020

Virtual Event, USA

Acceptance Rates

SoCC '20 Paper Acceptance Rate 35 of 143 submissions, 24%;

Overall Acceptance Rate 169 of 722 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
1,644
Total Downloads

Downloads (Last 12 months)273
Downloads (Last 6 weeks)39

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cvetković LCosta FDjokic MFriedman MKlimovic AWitchel EArpaci-Dusseau ARossbach CKeeton K(2024)Dirigent: Lightweight Serverless OrchestrationProceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles10.1145/3694715.3695966(369-384)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3694715.3695966
Szekely ABelay AMorris RKaashoek MWitchel EArpaci-Dusseau ARossbach CKeeton K(2024)Unifying serverless and microservice workloads with SigmaOSProceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles10.1145/3694715.3695947(385-402)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3694715.3695947
Liu YGuo JJiang BZhang PSun XSong YRen WHou ZLyu BWen RZhu SWang XVallina-Rodríguez NSuarez-Tángil GLevin DPelsser C(2024)Understanding Network Startup for Secure Containers in Multi-Tenant Clouds: Performance, Bottleneck and OptimizationProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688436(635-650)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3646547.3688436
Pan SZhao HCai ZLi DMa RGuan H(2024)Sustainable Serverless Computing with Cold-start Optimization and Automatic Workflow Resource SchedulingIEEE Transactions on Sustainable Computing10.1109/TSUSC.2023.3311197(1-12)Online publication date: 2024
https://doi.org/10.1109/TSUSC.2023.3311197
Qi SMonis LZeng ZWang IRamakrishnan K(2024)SPRIGHT: High-Performance eBPF-Based Event-Driven, Shared-Memory Processing for Serverless ComputingIEEE/ACM Transactions on Networking10.1109/TNET.2024.336656132:3(2539-2554)Online publication date: Jun-2024
https://doi.org/10.1109/TNET.2024.3366561
Lin FZhang XChen GChen LLi KJiang H(2024)Slim and Fast: Low-Overhead Container Overlay Network With Fast Connection SetupIEEE Transactions on Cloud Computing10.1109/TCC.2023.328223812:1(1-12)Online publication date: Jan-2024
https://doi.org/10.1109/TCC.2023.3282238
Liang HZhang ZHu CGong YCheng D(2024)A Survey on Spatio-Temporal Big Data Analytics Ecosystem: Resource Management, Processing Platform, and ApplicationsIEEE Transactions on Big Data10.1109/TBDATA.2023.334261910:2(174-193)Online publication date: Apr-2024
https://doi.org/10.1109/TBDATA.2023.3342619
Lee KKim JKwon IPark HHong C(2023)Impact of Secure Container Runtimes on File I/O Performance in Edge ComputingApplied Sciences10.3390/app13241332913:24(13329)Online publication date: 18-Dec-2023
https://doi.org/10.3390/app132413329
Joosen AHassan AAsenov MSingh RDarlow LWang JBarker A(2023)How Does It Function? Characterizing Long-term Trends in Production Serverless WorkloadsProceedings of the 2023 ACM Symposium on Cloud Computing10.1145/3620678.3624783(443-458)Online publication date: 30-Oct-2023
https://dl.acm.org/doi/10.1145/3620678.3624783
Kharbanda SFonseca PUstiugov DBruno RFonseca PGrot BBarbalace A(2023)Always-On Recording Framework for Serverless Computations: Opportunities and ChallengesProceedings of the 1st Workshop on SErverless Systems, Applications and MEthodologies10.1145/3592533.3592810(41-49)Online publication date: 8-May-2023
https://dl.acm.org/doi/10.1145/3592533.3592810
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents