Machine Learning Workflows in the Computing Continuum for Environmental Monitoring
Authors: 
Alessio Catalfamo, 
Atakan Aral, Ivona Brandic, Ewa Deelman, Massimo VillariAuthors Info & Claims
Pages 368 - 382
Abstract
Cloud-Edge Continuum is an innovative approach that exploits the strengths of the two paradigms: Cloud and Edge computing. This new approach gives us a holistic vision of this environment, enabling new kinds of applications that can exploit both the Edge computing advantages (e.g., real-time response, data security, and so on) and the powerful Cloud computing infrastructure for high computational requirements.
This paper proposes a Cloud-Edge computing Workflow solution for Machine Learning (ML) inference in a hydrogeological use case. Our solution is designed in a Cloud-Edge Continuum environment thanks to Pegasus Workflow Management System Tools that we use for the implementation phase. The proposed work splits the inference tasks, transparently distributing the computation performed by each layer between Cloud and Edge infrastructure. We use two models to implement a proof-of-concept of the proposed solution.
References
[1]
Cao K, Liu Y, Meng G, and Sun Q An overview on edge computing research IEEE Access 2020 8 85714-85728
[2]
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition (2015)
[3]
Hu, C., Li, B.: Distributed inference with deep learning models across heterogeneous edge devices. In: IEEE Conference on Computer Communications (IEEE INFOCOM 2022), pp. 330–339 (2022)
[4]
Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks (2018)
[5]
James, N., Ong, L.-Y., Leow, M.-C.: Exploring distributed deep learning inference using raspberry pi spark cluster. Future Internet 14(8), 220 (2022).
[6]
Jin, T., Hong, S.: Split-cnn: splitting window-based operations in convolutional neural networks for memory system optimization. In: Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS 2019), pp. 835–847. Association for Computing Machinery, New York (2019)
[7]
Klingler C, Schulz K, and Herrnegger M Lamah-CE: large-sample data for hydrology and environmental sciences for central Europe Earth Syst. Sci. Data 2021 13 9 4529-4565
[8]
Luger, D., Aral, A., Brandic, I.: Cost-aware neural network splitting and dynamic rescheduling for edge intelligence. In: Proceedings of the 6th International Workshop on Edge Systems, Analytics and Networking (EdgeSys 2023), pp. 42–47. Association for Computing Machinery, New York (2023)
[9]
MalekHosseini, E., Hajabdollahi, M., Karimi, N., Samavi, S., Shirani, S.: Splitting convolutional neural network structures for efficient inference (2020)
[10]
Matsubara, Y., Baidya, S., Callegaro, D., Levorato, M., Singh, S.: Distilled split deep neural networks for edge-assisted real-time systems. In: Proceedings of the 2019 Workshop on Hot Topics in Video Analytics and Intelligent Edges (2019)
[11]
Mehta, G., Deelman, E., Vahi, K., Silva, F.: Pegasus workflow management system: helping applications from earth and space. In: AGU Fall Meeting Abstracts, vol. 2010, pp. IN41B–1362 (2010)
[12]
Mehta, R., Shorey, R.: Deepsplit: dynamic splitting of collaborative edge-cloud convolutional neural networks. In: 2020 International Conference on COMmunication Systems and NETworkS (COMSNETS), pp. 720–725 (2020)
[13]
Parthasarathy, A., Krishnamachari, B.: Defer: distributed edge inference for deep neural networks. In: 2022 14th International Conference on COMmunication Systems and NETworkS (COMSNETS). IEEE (2022)
[14]
Pérez J, Díaz J, Berrocal J, López-Viana R, and González-Prieto A Edge computing: a grounded theory study Computing 2022 104 12 2711-2747
[15]
Redmon, J., Farhadi, A.: Yolo9000: Better, Faster, Stronger (2016)
[16]
Stahl, R., Zhao, Z., Mueller-Gritschneder, D., Gerstlauer, A., Schlichtmann, U.: Fully distributed deep learning inference on resource-constrained edge devices. In: Architectures, Modeling, and Simulation, Embedded Computer Systems (2019)
[17]
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., Wojna, Z.: Rethinking the inception architecture for computer vision (2015)
[18]
Teerapittayanon, S., McDanel, B., Kung, H.T.: Distributed deep neural networks over the cloud, the edge and end devices. In: Proceedings of the International Conference on Distributed Computing Systems, pp. 328–339 (2017)
[19]
Thomas, A., Guo, Y., Kim, Y., Aksanli, B., Kumar, A., Rosing, T.S.: Hierarchical and distributed machine learning inference beyond the edge. In: 2019 IEEE 16th International Conference on Networking, Sensing and Control (ICNSC), pp. 18–23 (2019)
[20]
Tian, Y., Zhang, Z., Yang, Z., Yang, Q.: JMSNAS: joint model split and neural architecture search for learning over mobile edge networks. In: 2022 IEEE International Conference on Communications Workshops (ICC Workshops), pp. 103–108 (2022)
[21]
Vaswani, A., et al.: Attention is all you need (2023)
[22]
Wen, Q., et al.: Transformers in time series: a survey (2023)
[23]
Yu, W., et al.: A survey on the edge computing for the internet of things. IEEE Access 6, 6900–6919 (2018)
Index Terms
- Machine Learning Workflows in the Computing Continuum for Environmental Monitoring
Index terms have been assigned to the content through auto-classification.
Recommendations
Workflow Engines in the Compute Continuum: a Comparative Analysis
UCC '23: Proceedings of the IEEE/ACM 16th International Conference on Utility and Cloud ComputingThe intersection of Cloud and Edge computing has ushered in a new era of distributed computing, known as Continuum Computing. This computational paradigm shift leverages the computational strength of Cloud infrastructure and the pervasiveness of Edge ...
Experiences using cloud computing for a scientific workflow application
ScienceCloud '11: Proceedings of the 2nd international workshop on Scientific cloud computingClouds are rapidly becoming an important platform for scientific applications. In this paper we describe our experiences running a scientific workflow application in the cloud. The application was developed to process astronomy data released by the ...
Reducing Time-to-Solution Using Distributed High-Throughput Mega-Workflows - Experiences from SCEC CyberShake
ESCIENCE '08: Proceedings of the 2008 Fourth IEEE International Conference on eScienceResearchers at the Southern California Earthquake Center (SCEC) use large-scale grid-based scientific workflows to perform seismic hazard research as a part of SCEC's program of earthquake system science research. The scientific goal of the SCEC ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Jul 2024
457 pages
ISBN:978-3-031-63774-2
DOI:10.1007/978-3-031-63775-9
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 02 July 2024
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 17 Jan 2025