More Web Proxy on the site http://driver.im/

research-article

Open access

FedZero: Leveraging Renewable Excess Energy in Federated Learning

Authors:

Philipp Wiesner,

Dennis Grinwald,

Pratik Agrawal,

Lauritz Thamsen,

Odej KaoAuthors Info & Claims

e-Energy '24: Proceedings of the 15th ACM International Conference on Future and Sustainable Energy Systems

Pages 373 - 385

https://doi.org/10.1145/3632775.3639589

Published: 21 May 2024 Publication History

All formats PDF

Abstract

Federated Learning (FL) is an emerging machine learning technique that enables distributed model training across data silos or edge devices without data sharing. Yet, FL inevitably introduces inefficiencies compared to centralized model training, which will further increase the already high energy usage and associated carbon emissions of machine learning in the future. One idea to reduce FL’s carbon footprint is to schedule training jobs based on the availability of renewable excess energy that can occur at certain times and places in the grid. However, in the presence of such volatile and unreliable resources, existing FL schedulers cannot always ensure fast, efficient, and fair training.

We propose FedZero, an FL system that operates exclusively on renewable excess energy and spare capacity of compute infrastructure to effectively reduce a training’s operational carbon emissions to zero. Using energy and load forecasts, FedZero leverages the spatio-temporal availability of excess resources by selecting clients for fast convergence and fair participation. Our evaluation, based on real solar and load traces, shows that FedZero converges significantly faster than existing approaches under the mentioned constraints while consuming less energy. Furthermore, it is robust to forecasting errors and scalable to tens of thousands of clients.

References

[1]

Ahmed M. Abdelmoniem, Atal Narayan Sahu, Marco Canini, and Suhaib A. Fahmy. 2023. REFL: Resource-Efficient Federated Learning. In EuroSys. ACM. https://doi.org/10.1145/3552326.3567485

Digital Library

[2]

David B. Alencar, Carolina de Mattos Affonso, Roberto C. L. Oliveira, Jorge Laureano Moya Rodríguez, Jandecy Cabral Leite, and Jose Carlos R. Filho. 2017. Different Models for Forecasting Wind Power Generation: Case Study. Energies 10 (2017). https://doi.org/10.3390/en10121976

[3]

Amazon. 2022. Amazon’s 2022 Sustainability Report. (2022).

[4]

Amna Arouj and Ahmed M. Abdelmoniem. 2022. Towards Energy-Aware Federated Learning on Battery-Powered Clients. In Workshop on Data Privacy and Federated Learning Technologies for Mobile Edge Network at ACM MobiCom.

[5]

World Bank. 2022. State and Trends of Carbon Pricing 2022. Technical Report. Washington, DC: World Bank.

[6]

Noman Bashir, David Irwin, Prashant Shenoy, and Abel Souza. 2022. Sustainable Computing - Without the Hot Air. In HotCarbon.

[7]

Axel Berg, Mark O’Connor, and Miguel Tairum Cruz. 2021. Keyword Transformer: A Self-Attention Model for Keyword Spotting. In Proc. Interspeech 2021. 4249–4253. https://doi.org/10.21437/Interspeech.2021-1286

[8]

Daniel J Beutel, Taner Topal, Akhil Mathur, Xinchi Qiu, Titouan Parcollet, and Nicholas D Lane. 2020. Flower: A Friendly Federated Learning Research Framework. arXiv preprint arXiv:2007.14390 (2020).

[9]

Anders Bjørn, Shannon M. Lloyd, Matthew Brander, and H. Damon Matthews. 2022. Renewable energy certificates threaten the integrity of corporate science-based targets. Nature Climate Change 12, 6 (2022).

[10]

Keith Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloé Kiddon, Jakub Konečný, Stefano Mazzocchi, Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. 2019. Towards Federated Learning at Scale: System Design. In MLSys. https://proceedings.mlsys.org/paper/2019/file/bd686fd640be98efaae0091fa301e613-Paper.pdf

[11]

Jamie M. Bright, Sven Killinger, David Lingfors, and Nicholas A. Engerer. 2018. Improved satellite-derived PV power nowcasting using real-time power data from reference PV systems. Solar Energy 168 (2018). https://doi.org/10.1016/j.solener.2017.10.091

[12]

Sebastian Caldas, Sai Meher Karthik Duddu, Peter Wu, Tian Li, Jakub Konečnỳ, H Brendan McMahan, Virginia Smith, and Ameet Talwalkar. 2019. LEAF: A Benchmark for Federated Settings. In Workshop on Federated Learning for Data Privacy and Confidentiality at NeurIPS.

[13]

California ISO. 2024. Managing oversupply. http://www.caiso.com/informed/Pages/ManagingOversupply.aspx. accessed Jan. 2024.

[14]

Andrew Chien, Chaojie Zhang, Liuzixuan Lin, and Varsha Rao. 2022. Beyond PUE: Flexible Datacenters Empowering the Cloud to Decarbonize. In HotCarbon.

[15]

Andrew A Chien, Chaojie Zhang, and Hai Duc Nguyen. 2019. Zero-carbon Cloud: Research Challenges for Datacenters as Supply-following Loads. University of Chicago, Tech. Rep. CS-TR-2019-08 (2019).

[16]

Payal Dhar. 2020. The carbon impact of artificial intelligence. Nature Machine Intelligence 2 (2020), 423–425.

[17]

Jesse Dodge, Taylor Prewitt, Remi Tachet des Combes, Erika Odmark, Roy Schwartz, Emma Strubell, Alexandra Sasha Luccioni, Noah A. Smith, Nicole DeCario, and Will Buchanan. 2022. Measuring the Carbon Intensity of AI in Cloud Instances. In ACM FAccT. https://doi.org/10.1145/3531146.3533234

Digital Library

[18]

Jonatan Enes, Guillaume Fieni, Roberto R. Expósito, Romain Rouvoy, and Juan Touriño. 2020. Power Budgeting of Big Data Applications in Container-based Clusters. In IEEE CLUSTER.

[19]

Gilbert Fridgen, Marc-Fabian Körner, Steffen Walters, and Martin Weibelzahl. 2021. Not All Doom and Gloom: How Energy-Intensive and Temporally Flexible Data Center Applications May Actually Promote Renewable Energy Sources. Business & Information Systems Engineering 63, 3 (2021).

[20]

Google. 2022. 2022 Environmental Report. (2022).

[21]

Başak Güler and Aylin Yener. 2021. A Framework for Sustainable Federated Learning. In 2021 19th International Symposium on Modeling and Optimization in Mobile, Ad hoc, and Wireless Networks (WiOpt). https://doi.org/10.23919/WiOpt52861.2021.9589930

[22]

Harry Hsu, Hang Qi, and Matthew Brown. 2019. Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification. arXiv preprint arXiv:1909.06335 (2019).

[23]

Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q. Weinberger. 2017. Densely Connected Convolutional Networks. In CVPR.

[24]

Yae Jee Cho, Jianyu Wang, and Gauri Joshi. 2022. Towards Understanding Biased Client Selection in Federated Learning. In AISTATS.

[25]

Ji Chu Jiang, Burak Kantarci, Sema Oktug, and Tolga Soyata. 2020. Federated Learning in Smart City Sensing: Challenges and Opportunities. Sensors 20, 21 (2020).

[26]

Zhifeng Jiang, Wei Wang, Baochun Li, and Bo Li. 2022. Pisces: Efficient Federated Learning via Guided Asynchronous Training. In ACM Symposium on Cloud Computing (SoCC). https://doi.org/10.1145/3542929.3563463

Digital Library

[27]

Lucas Joppa. 2021. Made to measure: Sustainability commitment progress and updates. Microsoft. Retrieved Sept. 2023 from https://blogs.microsoft.com/blog/2021/07/14/made-to-measure-sustainability-commitment-progress-and-updates

[28]

Alexandra I. Khalyasmaa, Stanislav A. Eroshenko, T. Chakravarthy, Venu Gopal Gasi, Sandeep Kumar Yadav Bollu, Raphael Caire, Sai Kumar Reddy Atluri, and Suresh Karrolla. 2019. Prediction of Solar Power Generation Based on Random Forest Regressor Model. In IEEE SIBIRCON. https://doi.org/10.1109/SIBIRCON48586.2019.8958063

[29]

Alex Krizhevsky. 2009. Learning multiple layers of features from tiny images. Technical Report.

[30]

Fan Lai, Xiangfeng Zhu, Harsha V. Madhyastha, and Mosharaf Chowdhury. 2021. Oort: Efficient Federated Learning via Guided Participant Selection. In USENIX OSDI. https://www.usenix.org/conference/osdi21/presentation/lai

[31]

Chenning Li, Xiao Zeng, Mi Zhang, and Zhichao Cao. 2022. PyramidFL: A Fine-Grained Client Selection Framework for Efficient Federated Learning. In ACM MobiCom. https://doi.org/10.1145/3495243.3517017

Digital Library

[32]

Qing’an Li, Chang Cai, Yasunari Kamada, Takao Maeda, Yuto Hiromori, Shuni Zhou, and Jianzhong Xu. 2021. Prediction of power generation of two 30 kW Horizontal Axis Wind Turbines with Gaussian model. Energy 231 (2021). https://doi.org/10.1016/j.energy.2021.121075

[33]

Shaohong Li, Xi Wang, Xiao Zhang, Vasileios Kontorinis, Sreekumar Kodakara, David Lo, and Parthasarathy Ranganathan. 2020. Thunderbolt: Throughput-Optimized, Quality-of-Service-Aware Power Capping at Scale. In USENIX OSDI. https://www.usenix.org/conference/osdi20/presentation/li-shaohong

[34]

Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated Optimization in Heterogeneous Networks. In MLSys.

[35]

Liuzixuan Lin, Victor M. Zavala, and Andrew Chien. 2021. Evaluating Coupling Models for Cloud Datacenters and Power Grids. In ACM e-Energy. https://doi.org/10.1145/3447555.3464868

Digital Library

[36]

Longjun Liu, Hongbin Sun, Chao Li, Tao Li, Jingmin Xin, and Nanning Zheng. 2017. Managing Battery Aging for High Energy Availability in Green Datacenters. IEEE Transactions on Parallel and Distributed Systems 28, 12 (2017). https://doi.org/10.1109/TPDS.2017.2712778

Digital Library

[37]

H. B. McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. 2016. Communication-Efficient Learning of Deep Networks from Decentralized Data. In AISTATS.

[38]

Microsoft. 2022. 2022 Environmental Sustainability Report. (2022).

[39]

Rakshit Naidu, Harshita Diddee, Ajinkya K Mulay, Aleti Vardhan, Krithika Ramesh, and Ahmed Zamzam. 2021. Towards Quantifying the Carbon Emissions of Differentially Private Machine Learning. In Workshop on Socially Responsible Machine Learning at ICML.

[40]

Anh Nguyen, Tuong Do, Minh Tran, Binh X. Nguyen, Chien Duong, Tu Phan, Erman Tjiputra, and Quang D. Tran. 2022. Deep Federated Learning for Autonomous Driving. In 2022 IEEE Intelligent Vehicles Symposium (IV).

[41]

Jake Oster. 2022. How we count carbon emissions from electricity matters. Amazon. Retrieved Sept. 2023 from https://www.amazon.science/blog/how-we-count-carbon-emissions-from-electricity-matters

[42]

David Patterson, Joseph Gonzalez, Urs Hölzle, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David R. So, Maud Texier, and Jeff Dean. 2022. The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink. Computer 55, 7 (2022). https://doi.org/10.1109/MC.2022.3148714

Digital Library

[43]

Xinchi Qiu, Titouan Parcollet, Javier Fernandez-Marques, Pedro Porto Buarque de Gusmao, Daniel J. Beutel, Taner Topal, Akhil Mathur, and Nicholas D. Lane. 2021. A first look into the carbon footprint of federated learning. arXiv preprint arXiv:2102.07627 (2021).

[44]

Ana Radovanovic, Ross Koningstein, Ian Schneider, Bokan Chen, Alexandre Duarte, Binz Roy, Diyue Xiao, Maya Haridasan, Patrick Hung, Nick Care, Saurav Talukdar, Eric Mullen, Kendal Smith, Mariellen Cottman, and Walfredo Cirne. 2022. Carbon-Aware Computing for Datacenters. IEEE Transactions on Power Systems (2022).

[45]

Martin Rapp, Ramin Khalili, Kilian Pfeiffer, and Jörg Henkel. 2022. DISTREAL: Distributed Resource-Aware Learning in Heterogeneous Systems. In AAAI.

[46]

REN21. 2022. Renewables 2022 Global Status Report. (2022).

[47]

Nicola Rieke, Jonny Hancox, Wenqi Li, Fausto Milletarì, Holger R. Roth, Shadi Albarqouni, Spyridon Bakas, Mathieu N. Galtier, Bennett A. Landman, Klaus Maier-Hein, Sébastien Ourselin, Micah Sheller, Ronald M. Summers, Andrew Trask, Daguang Xu, Maximilian Baust, and M. Jorge Cardoso. 2020. The future of digital health with federated learning. npj Digital Medicine 3, 1 (2020).

[48]

René Schwermer, Ruben Mayer, and Hans-Arno Jacobsen. 2023. Energy vs Privacy: Estimating the Ecological Impact of Federated Learning. In ACM e-Energy.

[49]

Jinhyun So, Kevin Hsieh, Behnaz Arzani, Shadi Noghabi, Salman Avestimehr, and Ranveer Chandra. 2022. FedSpace: An Efficient Federated Learning Framework at Satellites and Ground Stations. arXiv preprint arXiv:2202.01267 (2022).

[50]

Behnaz Soltani, Venus Haghighi, Adnan Mahmood, Quan Z. Sheng, and Lina Yao. 2022. A Survey on Participant Selection for Federated Learning in Mobile Networks. In Workshop on Mobility in the Evolving Internet Architecture (MobiArch) at MobiCom.

Digital Library

[51]

Abel Souza, Noman Bashir, Jorge Murillo, Walid Hanafy, Qianlin Liang, David Irwin, and Prashant Shenoy. 2023. Ecovisor: A Virtual Energy System for Carbon-Efficient Applications. In ASPLOS.

[52]

Emma Strubell, Ananya Ganesh, and Andrew McCallum. 2020. Energy and Policy Considerations for Modern Deep Learning Research. In AAAI.

[53]

Mingxing Tan and Quoc Le. 2019. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In ICML.

[54]

Maud Texier. 2021. A timely new approach to certifying clean energy. Google. Retrieved Sept. 2023 from https://cloud.google.com/blog/topics/sustainability/t-eacs-offer-new-approach-to-certifying-clean-energy

[55]

Muhammad Tirmazi, Adam Barker, Nan Deng, Md E. Haque, Zhijing Gene Qin, Steven Hand, Mor Harchol-Balter, and John Wilkes. 2020. Borg: The next Generation. In EuroSys.

Digital Library

[56]

Cong Wang, Bin Hu, and Hongyi Wu. 2022. Energy Minimization for Federated Asynchronous Learning on Battery-Powered Mobile Devices via Application Co-running. In ICDCS.

[57]

Qizhen Weng, Wencong Xiao, Yinghao Yu, Wei Wang, Cheng Wang, Jian He, Yong Li, Liping Zhang, Wei Lin, and Yu Ding. 2022. MLaaS in the Wild: Workload Analysis and Scheduling in Large-Scale Heterogeneous GPU Clusters. In USENIX NSDI.

[58]

Philipp Wiesner, Ilja Behnke, and Odej Kao. 2023. A Testbed for Carbon-Aware Applications and Systems. arxiv:2306.09774 [cs.DC]

[59]

Philipp Wiesner, Ilja Behnke, Dominik Scheinert, Kordian Gontarska, and Lauritz Thamsen. 2021. Let’s Wait Awhile: How Temporal Workload Shifting Can Reduce Carbon Emissions in the Cloud. In ACM Middleware.

[60]

Philipp Wiesner, Dominik Scheinert, Thorsten Wittkopp, Lauritz Thamsen, and Odej Kao. 2022. Cucumber: Renewable-Aware Admission Control for Delay-Tolerant Cloud and Edge Workloads. In International European Conference on Parallel and Distributed Computing (Euro-Par).

[61]

Carole-Jean Wu, Ramya Raghavendra, Udit Gupta, Bilge Acun, Newsha Ardalani, Kiwan Maeng, Gloria Chang, Fiona Aga Behram, Jinshi Huang, Charles Bai, Michael Gschwind, Anurag Gupta, Myle Ott, Anastasia Melnikov, Salvatore Candido, David Brooks, Geeta Chauhan, Benjamin Lee, Hsien-Hsin S. Lee, Bugra Akyildiz, Maximilian Balandat, Joe Spisak, Ravi Jain, Mike Rabbat, and Kim M. Hazelwood. 2022. Sustainable AI: Environmental Implications, Challenges and Opportunities. In MLSys.

[62]

Qiang Yang, Yang Liu, Yong Cheng, Yan Kang, Tianjian Chen, and Han Yu. 2019. Federated learning. Morgan & Claypool Publishers.

[63]

Zhaohui Yang, Mingzhe Chen, Walid Saad, Choong Seon Hong, and Mohammad Shikh-Bahaei. 2021. Energy Efficient Federated Learning Over Wireless Communication Networks. IEEE Transactions on Wireless Communications 20, 3 (2021).

Digital Library

[64]

Ashkan Yousefpour, Shen Guo, Ashish Shenoy, Sayan Ghosh, Pierre Stock, Kiwan Maeng, Schalk-Willem Krüger, Michael Rabbat, Carole-Jean Wu, and Ilya Mironov. 2023. Green Federated Learning. arxiv:2303.14604 [cs.LG]

[65]

Sai Qian Zhang, Jieyu Lin, and Qi Zhang. 2022. A Multi-Agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learning. In AAAI. https://doi.org/10.1609/aaai.v36i8.20894

[66]

Jiajia Zheng, Andrew A. Chien, and Sangwon Suh. 2020. Mitigating Curtailment and Carbon Emissions through Load Migration between Data Centers. Joule 4, 10 (2020).

[67]

Zhi Zhou, Fangming Liu, Yong Xu, Ruolan Zou, Hong Xu, John C.S. Lui, and Hai Jin. 2013. Carbon-Aware Load Balancing for Geo-distributed Cloud Services. In 21st Int. Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS).

[68]

Chen Zhu, Zheng Xu, Mingqing Chen, Jakub Konečný, Andrew Hard, and Tom Goldstein. 2022. Diurnal or Nocturnal? Federated Learning of Multi-branch Networks from Periodically Shifting Distributions. In ICLR.

Cited By

Liao HTang GGuo DWang YCao R(2024)Rethinking Low-Carbon Edge Computing System Design with Renewable Energy SharingProceedings of the 53rd International Conference on Parallel Processing10.1145/3673038.3673080(950-960)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3673038.3673080
Chaitanya Kumar MSatya Narayana J SBao YWang XDrew S(2024)Energy-efficient Federated Learning with Dynamic Model Size Allocation2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825664(8737-8741)Online publication date: 15-Dec-2024
https://doi.org/10.1109/BigData62323.2024.10825664
Iftikhar SDavy S(2024)Reducing Carbon Footprint in AI: A Framework for Sustainable Training of Large Language ModelsProceedings of the Future Technologies Conference (FTC) 2024, Volume 110.1007/978-3-031-73110-5_22(325-336)Online publication date: 5-Nov-2024
https://doi.org/10.1007/978-3-031-73110-5_22

Index Terms

FedZero: Leveraging Renewable Excess Energy in Federated Learning
1. Computing methodologies
  1. Artificial intelligence
    1. Distributed artificial intelligence
2. Social and professional topics
  1. Professional topics
    1. Computing industry
      1. Sustainability

Recommendations

The War of the Efficiencies: Understanding the Tension between Carbon and Energy Optimization
HotCarbon '23: Proceedings of the 2nd Workshop on Sustainable Computer Systems

Major innovations in computing have been driven by scaling up computing infrastructure, while aggressively optimizing operating costs. The result is a network of worldwide datacenters that consume a large amount of energy, mostly in an energy-...
The War of the Efficiencies: Understanding the Tension between Carbon and Energy Optimization

Major innovations in computing have been driven by scaling up computing infrastructure, while aggressively optimizing operating costs. The result is a network of worldwide datacenters that consume a large amount of energy, mostly in an energy-efficient ...
A first look into the carbon footprint of federated learning

Despite impressive results, deep learning-based technologies also raise severe privacy and environmental concerns induced by the training procedure often conducted in data centers. In response, alternatives to centralized training such as Federated ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

e-Energy '24: Proceedings of the 15th ACM International Conference on Future and Sustainable Energy Systems

June 2024

704 pages

ISBN:9798400704802

DOI:10.1145/3632775

Copyright © 2024 Owner/Author.

This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 May 2024

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Conference

e-Energy '24

e-Energy '24: The 15th ACM International Conference on Future and Sustainable Energy Systems

June 4 - 7, 2024

Singapore, Singapore

Acceptance Rates

Overall Acceptance Rate 160 of 446 submissions, 36%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
476
Total Downloads

Downloads (Last 12 months)476
Downloads (Last 6 weeks)52

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liao HTang GGuo DWang YCao R(2024)Rethinking Low-Carbon Edge Computing System Design with Renewable Energy SharingProceedings of the 53rd International Conference on Parallel Processing10.1145/3673038.3673080(950-960)Online publication date: 12-Aug-2024
https://dl.acm.org/doi/10.1145/3673038.3673080
Chaitanya Kumar MSatya Narayana J SBao YWang XDrew S(2024)Energy-efficient Federated Learning with Dynamic Model Size Allocation2024 IEEE International Conference on Big Data (BigData)10.1109/BigData62323.2024.10825664(8737-8741)Online publication date: 15-Dec-2024
https://doi.org/10.1109/BigData62323.2024.10825664
Iftikhar SDavy S(2024)Reducing Carbon Footprint in AI: A Framework for Sustainable Training of Large Language ModelsProceedings of the Future Technologies Conference (FTC) 2024, Volume 110.1007/978-3-031-73110-5_22(325-336)Online publication date: 5-Nov-2024
https://doi.org/10.1007/978-3-031-73110-5_22

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten