More Web Proxy on the site http://driver.im/

research-article

Open access

Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices

Authors:

Jiang XieAuthors Info & Claims

SEC '23: Proceedings of the Eighth ACM/IEEE Symposium on Edge Computing

Pages 80 - 93

https://doi.org/10.1145/3583740.3628442

Published: 07 August 2024 Publication History

Abstract

Today, deep learning optimization is primarily driven by research focused on achieving high inference accuracy and reducing latency. However, the energy efficiency aspect is often overlooked, possibly due to a lack of sustainability mindset in the field and the absence of a holistic energy dataset. In this paper, we conduct a threefold study, including energy measurement, prediction, and efficiency scoring, with an objective to foster transparency in power and energy consumption within deep learning across various edge devices. Firstly, we present a detailed, first-of-its-kind measurement study that uncovers the energy consumption characteristics of on-device deep learning. This study results in the creation of three extensive energy datasets for edge devices, covering a wide range of kernels, state-of-the-art DNN models, and popular AI applications. Secondly, we design and implement the first kernel-level energy predictors for edge devices based on our kernel-level energy dataset. Evaluation results demonstrate the ability of our predictors to provide consistent and accurate energy estimations on unseen DNN models. Lastly, we introduce two scoring metrics, PCS and IECS, developed to convert complex power and energy consumption data of an edge device into an easily understandable manner for edge device end-users. We hope our work can help shift the mindset of both end-users and the research community towards sustainability in edge computing, a principle that drives our research. Find data, code, and more up-to-date information at https://amai-gsu.github.io/DeepEn2023.

References

[1]

Andrew G Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861, 2017.

[2]

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84--90, 2017.

Digital Library

[3]

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception architecture for computer vision. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2818--2826, 2016.

[4]

Haoxiang Li, Zhe Lin, Xiaohui Shen, Jonathan Brandt, and Gang Hua. A convolutional neural network cascade for face detection. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5325--5334, 2015.

[5]

Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 815--823, 2015.

[6]

Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 779--788, 2016.

[7]

Jonathan Huang, Vivek Rathod, Chen Sun, Menglong Zhu, Anoop Korattikara, Alireza Fathi, Ian Fischer, Zbigniew Wojna, Yang Song, Sergio Guadarrama, et al. Speed/accuracy trade-offs for modern convolutional object detectors. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 7310--7311, 2017.

[8]

Y. Wu, J. Lim, and M. Yang. Object tracking benchmark. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9):1834--1848, 2015.

Digital Library

[9]

Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(2):295--307, 2015.

Digital Library

[10]

Christian Ledig, Lucas Theis, Ferenc Huszár, Jose Caballero, Andrew Cunningham, Alejandro Acosta, Andrew Aitken, Alykhan Tejani, Johannes Totz, Zehan Wang, et al. Photo-realistic single image super-resolution using a generative adversarial network. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 4681--4690, 2017.

[11]

Radu Timofte, Shuhang Gu, Jiqing Wu, and Luc Van Gool. Ntire 2018 challenge on single image super-resolution: Methods and results. In Proc. IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 852--863, 2018.

[12]

Liang-Chieh Chen, Yukun Zhu, George Papandreou, Florian Schroff, and Hartwig Adam. Encoder-decoder with atrous separable convolution for semantic image segmentation. In Proc. European Conference on Computer Vision (ECCV), pages 801--818, 2018.

Digital Library

[13]

George Papandreou, Tyler Zhu, Liang-Chieh Chen, Spyros Gidaris, Jonathan Tompson, and Kevin Murphy. Personlab: Person pose estimation and instance segmentation with a bottom-up, part-based, geometric embedding model. In Proc. European Conference on Computer Vision (ECCV), pages 269--286, 2018.

Digital Library

[14]

Francisco Javier Ordóñez and Daniel Roggen. Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition. Sensors, 16(1):115, 2016.

[15]

Iulian V Serban, Chinnadhurai Sankar, Mathieu Germain, Saizheng Zhang, Zhouhan Lin, Sandeep Subramanian, Taesup Kim, Michael Pieper, Sarath Chandar, Nan Rosemary Ke, et al. A deep reinforcement learning chatbot. arXiv preprint arXiv:1709.02349, 2017.

[16]

Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint arXiv:1606.05250, 2016.

[17]

Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473, 2014.

[18]

Ilya Sutskever, Oriol Vinyals, and Quoc V Le. Sequence to sequence learning with neural networks. arXiv preprint arXiv:1409.3215, 2014.

[19]

Aliaksei Severyn and Alessandro Moschitti. Twitter sentiment analysis with deep convolutional neural networks. In Proc. the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 959--962, 2015.

Digital Library

[20]

Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proc. Conference on Empirical Methods in Natural Language Processing, pages 1631--1642, 2013.

[21]

Veton Kepuska and Gamal Bohouta. Next-generation of virtual personal assistants (microsoft cortana, apple siri, amazon alexa and google home). In Proc. IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), pages 99--103, 2018.

[22]

Chung-Cheng Chiu, Tara N Sainath, Yonghui Wu, Rohit Prabhavalkar, Patrick Nguyen, Zhifeng Chen, Anjuli Kannan, Ron J Weiss, Kanishka Rao, Ekaterina Gonina, et al. State-of-the-art speech recognition with sequence-to-sequence models. In Proc. 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 4774--4778, 2018.

Digital Library

[23]

Haomin Zhang, Ian McLoughlin, and Yan Song. Robust sound event recognition using convolutional neural networks. In Proc. 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 559--563, 2015.

[24]

Haoxin Wang, BaekGyu Kim, Jiang Xie, and Zhu Han. LEAF + AIO: Edge-assisted energy-aware object detection for mobile augmented reality. IEEE Transactions on Mobile Computing, 22(10):5933--5948, 2023.

Digital Library

[25]

Haoxin Wang and Jiang Xie. User preference based energy-aware mobile AR system with edge computing. In Proc. IEEE Conference on Computer Communications (INFOCOM), pages 1379--1388, 2020.

Digital Library

[26]

Users Reveal Top Frustrations That Lead to Bad Mobile App Reviews. https://finance.yahoo.com/news/apigee-survey-users-reveal-top-120200656. Accessed on March 2023.

[27]

Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et al. TVM: An automated end-to-end optimizing compiler for deep learning. In Proc. 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI), pages 578--594, 2018.

[28]

Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. Tensorflow: a system for large-scale machine learning. In Proc. 12th USENIX symposium on operating systems design and implementation (OSDI), pages 265--283, 2016.

[29]

Xiaotang Jiang, Huan Wang, Yiliu Chen, Ziqi Wu, Lichuan Wang, Bin Zou, Yafeng Yang, Zongyang Cui, Yu Cai, Tianhang Yu, et al. MNN: A universal and efficient inference engine. In Proc. Machine Learning and Systems (MLSys), pages 1--13, 2020.

[30]

Li Lyna Zhang, Shihao Han, Jianyu Wei, Ningxin Zheng, Ting Cao, Yuqing Yang, and Yunxin Liu. NN-meter: Towards accurate latency prediction of deep-learning model inference on diverse edge devices. In Proc. the 19th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys), pages 81--93, 2021.

Digital Library

[31]

Han Cai, Ligeng Zhu, and Song Han. ProxylessNAS: Direct neural architecture search on target task and hardware. In Proc. International Conference on Learning Representations (ICLR), 2019.

[32]

Xiaoliang Dai, Peizhao Zhang, Bichen Wu, Hongxu Yin, Fei Sun, Yanghan Wang, Marat Dukhan, Yunqing Hu, Yiming Wu, Yangqing Jia, et al. Chamnet: Towards efficient network design through platform-aware model adaptation. In Proc. the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11398--11407, 2019.

[33]

Monsoon Power Monitor. https://www.msoon.com/specifications. Accessed on March 2023.

[34]

Abhilash Jindal and Y Charlie Hu. Experience: developing a usable battery drain testing and diagnostic tool for the mobile industry. In Proc. the 27th Annual International Conference on Mobile Computing and Networking (MobiCom), pages 804--815, 2021.

[35]

Kittipat Apicharttrisorn, Xukan Ran, Jiasi Chen, Srikanth V Krishnamurthy, and Amit K Roy-Chowdhury. Frugal following: Power thrifty object detection and tracking for mobile augmented reality. In Proc. the 17th Conference on Embedded Networked Sensor Systems (SenSys), pages 96--109, 2019.

Digital Library

[36]

Xukan Ran, Haolianz Chen, Xiaodan Zhu, Zhenming Liu, and Jiasi Chen. Deep-decision: A mobile deep learning framework for edge video analytics. In Proc. IEEE Conference on Computer Communications (INFOCOM), pages 1421--1429, 2018.

Digital Library

[37]

Xiaomeng Chen, Abhilash Jindal, Ning Ding, Yu Charlie Hu, Maruti Gupta, and Rath Vannithamby. Smartphone background activities in the wild: Origin, energy drain, and optimization. In Proc. the 21st Annual International Conference on Mobile Computing and Networking, pages 40--52, 2015.

Digital Library

[38]

Abhinav Pathak, Y Charlie Hu, and Ming Zhang. Where is the energy spent inside my app? fine grained energy accounting on smartphones with Eprof. In Proc. the 7th ACM European Conference on Computer Systems (EuroSys), pages 29--42, 2012.

Digital Library

[39]

Anik Mallik, Haoxin Wang, Jiang Xie, Dawei Chen, and Kyungtae Han. EPAM: A predictive energy model for mobile AI. In Proc. IEEE International Conference on Communications (ICC), pages 1--6, 2023.

[40]

Abram Hindle, Alex Wilson, Kent Rasmussen, E Jed Barlow, Joshua Charles Campbell, and Stephen Romansky. Greenminer: A hardware based mining software repositories software energy consumption framework. In Proc. the 11th ACM Working Conference on Mining Software Repositories, pages 12--21, 2014.

Digital Library

[41]

Haoxin Wang, BaekGyu Kim, Jiang Xie, and Zhu Han. Energy drain of the object detection processing pipeline for mobile devices: Analysis and implications. IEEE Transactions on Green Communications and Networking, 5(1):41--60, 2021.

[42]

Andrea McIntosh, Safwat Hassan, and Abram Hindle. What can android mobile app developers do about the energy consumption of machine learning? Empirical Software Engineering, 24:562--601, 2019.

Digital Library

[43]

Haoxin Wang, BaekGyu Kim, Jiang Xie, and Zhu Han. How is energy consumed in smartphone deep learning apps? executing locally vs. remotely. In Proc. IEEE Global Communications Conference (GLOBECOM), pages 1--6, 2019.

Digital Library

[44]

Andrey Ignatov, Radu Timofte, William Chou, Ke Wang, Max Wu, Tim Hartley, and Luc Van Gool. AI benchmark: Running deep neural networks on android smartphones. In Proc. the European Conference on Computer Vision (ECCV) Workshops, 2018.

[45]

Mechanic Mobile Device DC Power Cable. https://www.amazon.com/Mechanic-Supply-Mobile-Repair-Control/dp/B089F1PM1F. Accessed on March 2023.

[46]

TensorFlow Performance Measurement. https://www.tensorflow.org/lite/performance/measurement. Accessed on March 2023.

[47]

Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: Enhanced super-resolution generative adversarial networks. In Proc. European Conference on Computer Vision (ECCV) Workshops, 2018.

[48]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.

[49]

Pete Warden. Speech commands: A dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209, 2018.

[50]

Radosvet Desislavov, Fernando Martínez-Plumed, and José Hernández-Orallo. Trends in AI inference energy consumption: Beyond the performance-vs-parameter laws of deep learning. Sustainable Computing: Informatics and Systems, 38:100857, 2023.

[51]

Ning Ding and Y Charlie Hu. GfxDoctor: A holistic graphics energy profiler for mobile devices. In Proc. the Twelfth European Conference on Computer Systems, pages 359--373, 2017.

Digital Library

[52]

Andrey Ignatov, Radu Timofte, Andrei Kulik, Seungsoo Yang, Ke Wang, Felix Baum, Max Wu, Lirong Xu, and Luc Van Gool. AI benchmark: All about deep learning on smartphones in 2019. In Proc. 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), pages 3617--3635, 2019.

[53]

Vijay Janapa Reddi, Christine Cheng, David Kanter, Peter Mattson, Guenther Schmuelling, Carole-Jean Wu, Brian Anderson, Maximilien Breughe, Mark Charlebois, William Chou, et al. Mlperf inference benchmark. In Proc. ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA), pages 446--459, 2020.

Digital Library

[54]

Vijay Janapa Reddi, David Kanter, Peter Mattson, Jared Duke, Thai Nguyen, Ramesh Chukka, Ken Shiring, Koan-Sin Tan, Mark Charlebois, William Chou, et al. Mlperf mobile inference benchmark: An industry-standard open-source machine learning benchmark for on-device AI. In Proc. Machine Learning and Systems (MLSys), volume 4, pages 352--369, 2022.

[55]

Chunjie Luo, Xiwen He, Jianfeng Zhan, Lei Wang, Wanling Gao, and Jiahui Dai. Comparison and benchmarking of AI models and frameworks on mobile devices. arXiv preprint arXiv:2005.05085, 2020.

Cited By

Chen YZhang QXing RLi YMa XYu CZhang YZhou AWang S(2024)Energy-Aware Satellite-Ground Co-Inference via Layer-Wise Processing Schedule OptimizationProceedings of the 15th Asia-Pacific Symposium on Internetware10.1145/3671016.3674811(303-312)Online publication date: 24-Jul-2024
https://dl.acm.org/doi/10.1145/3671016.3674811
Mao YYu XHuang KAngela Zhang YZhang J(2024)Green Edge AI: A Contemporary SurveyProceedings of the IEEE10.1109/JPROC.2024.3437365112:7(880-911)Online publication date: Jul-2024
https://doi.org/10.1109/JPROC.2024.3437365
Mallik AXie JHan Z(2024)A Performance Analysis Modeling Framework for Extended Reality Applications in Edge-Assisted Wireless Networks2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS60910.2024.00073(726-737)Online publication date: 23-Jul-2024
https://doi.org/10.1109/ICDCS60910.2024.00073
Show More Cited By

Index Terms

Unveiling Energy Efficiency in Deep Learning: Measurement, Prediction, and Scoring across Edge Devices
1. Computer systems organization
  1. Embedded and cyber-physical systems
2. Computing methodologies
  1. Machine learning

Recommendations

Energy Consumption of IT System in Cloud Data Center: Architecture, Factors and Prediction
Network and Parallel Computing
Abstract
In recent years, as cloud data center has grown constantly in size and quantity, the energy consumption of cloud data center has increased dramatically. Therefore, it is of great significance to study the energy-saving issues of cloud data centers ...
Energy Efficiency Trade-Off Between Duty-Cycling and Wake-Up Radio Techniques in IoT Networks
Abstract
Energy consumption has become dominant issue for wireless internet of things (IoT) networks with battery-powered nodes. The prevailing mechanism allowing to reduce energy consumption is duty-cycling. In this technique the node sleeps most of the ...
Sustainable Connections: Exploring Energy Efficiency in 5G Networks
CoNEXT '24: Proceedings of the 20th International Conference on emerging Networking EXperiments and Technologies

Although 5G networks offer larger capacity due to more antennas and larger bandwidths, their increased energy consumption is concerning. This paper investigates energy consumption issues from widespread 5G deployment using city-scale real-world mobile ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

SEC '23: Proceedings of the Eighth ACM/IEEE Symposium on Edge Computing

December 2023

405 pages

ISBN:9798400701238

DOI:10.1145/3583740

Chair:
Kewei Sha,
Program Chairs:
Suman Banerjee,
Jiasi Chen
University of Michigan

This work is licensed under a Creative Commons Attribution International 4.0 License.

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing
IEEE Computer Society

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 August 2024

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

US National Science Foundation (NSF)
Toyota Motor North America

Conference

SEC '23

Sponsor:

SIGMOBILE

SEC '23: Eighth ACM/IEEE Symposium on Edge Computing

December 6 - 9, 2023

DE, Wilmington, USA

Acceptance Rates

Overall Acceptance Rate 40 of 100 submissions, 40%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
107
Total Downloads

Downloads (Last 12 months)107
Downloads (Last 6 weeks)45

Reflects downloads up to 19 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Chen YZhang QXing RLi YMa XYu CZhang YZhou AWang S(2024)Energy-Aware Satellite-Ground Co-Inference via Layer-Wise Processing Schedule OptimizationProceedings of the 15th Asia-Pacific Symposium on Internetware10.1145/3671016.3674811(303-312)Online publication date: 24-Jul-2024
https://dl.acm.org/doi/10.1145/3671016.3674811
Mao YYu XHuang KAngela Zhang YZhang J(2024)Green Edge AI: A Contemporary SurveyProceedings of the IEEE10.1109/JPROC.2024.3437365112:7(880-911)Online publication date: Jul-2024
https://doi.org/10.1109/JPROC.2024.3437365
Mallik AXie JHan Z(2024)A Performance Analysis Modeling Framework for Extended Reality Applications in Edge-Assisted Wireless Networks2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS60910.2024.00073(726-737)Online publication date: 23-Jul-2024
https://doi.org/10.1109/ICDCS60910.2024.00073
Kasioulis MSymeonides MIoannou GPallis GDikaiakos M(2024)Energy modeling of inference workloads with AI accelerators at the Edge: A benchmarking study2024 IEEE International Conference on Cloud Engineering (IC2E)10.1109/IC2E61754.2024.00028(189-196)Online publication date: 24-Sep-2024
https://doi.org/10.1109/IC2E61754.2024.00028

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents