More Web Proxy on the site http://driver.im/

research-article

Learning from Failures: Secure and Fault-Tolerant Aggregation for Federated Learning

Authors:

Mohamad Mansouri,

Wafa Ben JaballahAuthors Info & Claims

ACSAC '22: Proceedings of the 38th Annual Computer Security Applications Conference

Pages 146 - 158

https://doi.org/10.1145/3564625.3568135

Published: 05 December 2022 Publication History

Abstract

Federated learning allows multiple parties to collaboratively train a global machine learning (ML) model without sharing their private datasets. To make sure that these local datasets are not leaked, existing works propose to rely on a secure aggregation scheme that allows parties to encrypt their model updates before sending them to the central server that aggregates the encrypted inputs. In this work, we design and evaluate a new secure and fault-tolerant aggregation scheme for federated learning that is robust against client failures. We first develop a threshold-variant of the secure aggregation scheme proposed by Joye and Libert. Using this new building block together with a dedicated decentralized key management scheme and an input encoding solution, we design a privacy-preserving federated learning protocol that, when executed among n clients, can recover from up to failures. Our solution is secure against a malicious aggregator who can manipulate messages to learn clients’ individual inputs. We show that our solution outperforms the state-of-the-art fault-tolerant secure aggregation schemes in terms of computation cost on the client. For example, with an ML model of 100,000 parameters, trained with 600 clients, our protocol is 5.5x faster (1.6x faster in case of 180 clients drop).

References

[1]

Constance Beguier and Eric W. Tramel. 2020. SAFER: Sparse Secure Aggregation for Federated Learning. arxiv:2007.14861 [stat.ML]

[2]

James Henry Bell, Kallista A. Bonawitz, Adrià Gascón, Tancrède Lepoint, and Mariana Raykova. 2020. Secure Single-Server Aggregation with (Poly)Logarithmic Overhead. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security(CCS ’20). Association for Computing Machinery.

Digital Library

[3]

Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. 2017. Practical Secure Aggregation for Privacy-Preserving Machine Learning(CCS ’17). Association for Computing Machinery, New York, NY, USA, 1175–1191.

[4]

Keith Bonawitz, Fariborz Salehi, Jakub Konečný, Brendan McMahan, and Marco Gruteser. 2019. Federated Learning with Autotuned Communication-Efficient Secure Aggregation. In 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[5]

Kallista A. Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloé Kiddon, Jakub Konečný, Stefano Mazzocchi, H. Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. 2019. Towards Federated Learning at Scale: System Design. CoRR abs/1902.01046(2019).

[6]

Dan Boneh and Matthew Franklin. 2001. Efficient Generation of Shared RSA Keys. J. ACM 48, 4 (jul 2001), 702–722. https://doi.org/10.1145/502090.502094

Digital Library

[7]

Léon Bottou. 2004. Stochastic Learning. In Advanced Lectures on Machine Learning, Olivier Bousquet and Ulrike von Luxburg (Eds.). Springer Verlag, Berlin, 146–168. http://leon.bottou.org/papers/bottou-mlss-2004

[8]

Megan Chen, Jack Doerner, Yashvanth Kondi, Eysa Lee, Schuyler Rosefield, Abhi Shelat, and Ran Cohen. 2022. Multiparty Generation of an RSA Modulus. Journal of Cryptology 35, 2 (2022).

Digital Library

[9]

Ivan Damgård, Mads Jurik, and Jesper Buus Nielsen. 2010. A generalization of Paillier’s public-key system with applications to electronic voting. International Journal of Information Security 9, 6 (01 Dec 2010).

Digital Library

[10]

Tassos Dimitriou and Mohamad Khattar Awad. 2016. Secure and scalable aggregation in the smart grid resilient against malicious entities. Ad Hoc Networks 50(2016).

[11]

Ye Dong, Xiaojun Chen, Liyan Shen, and Dakui Wang. 2020. EaSTFLy: Efficient and secure ternary federated learning. Computers & Security 94(2020), 101824.

[12]

Cynthia Dwork. 2006. Differential Privacy. In Automa, Languages and Programming. Springer Berlin Heidelberg.

[13]

Morris Dworkin, Elaine Barker, James Nechvatal, James Foti, Lawrence Bassham, E. Roback, and James Dray. 2001. Advanced Encryption Standard (AES).

[14]

Morris J. Dworkin. 2007. SP 800-38D. Recommendation for Block Cipher Modes of Operation: Galois/Counter Mode (GCM) and GMAC. Technical Report. Gaithersburg, MD, USA.

[15]

Ahmed Roushdy Elkordy and A. Salman Avestimehr. 2020. Secure Aggregation with Heterogeneous Quantization in Federated Learning.

[16]

Zekeriya Erkin and Gene Tsudik. 2012. Private Computation of Spatial and Temporal Power Consumption with Smart Meters. In Applied Cryptography and Network Security. Springer Berlin Heidelberg, 561–577.

[17]

Xiaojie Guo, Zheli Liu, Jin Li, Jiqiang Gao, Boyu Hou, Changyu Dong, and Thar Baker. 2021. VeriFL: Communication-Efficient and Fast Verifiable Aggregation for Federated Learning. IEEE Transactions on Information Forensics and Security 16 (2021).

[18]

Marc Joye and Benoît Libert. 2013. A Scalable Scheme for Privacy-Preserving Aggregation of Time-Series Data. In Financial Cryptography and Data Security, Ahmad-Reza Sadeghi (Ed.). Springer Berlin Heidelberg.

[19]

Swanand Kadhe, Nived Rajaraman, Onur Ozan Koyluoglu, and Kannan Ramchandran. 2020. FastSecAgg: Scalable Secure Aggregation for Privacy-Preserving Federated Learning. CoRR abs/2009.11248(2020).

[20]

Peter Kairouz, Ziyu Liu, and Thomas Steinke. 2021. The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregation. In Proceedings of the 38th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 139). PMLR.

[21]

Ferhat Karakoç, Melek Önen, and Zeki Bilgin. 2021. Secure Aggregation Against Malicious Users. In Proceedings of the 26th ACM Symposium on Access Control Models and Technologies(SACMAT ’21). Association for Computing Machinery.

Digital Library

[22]

Youssef Khazbak, Tianxiang Tan, and Guohong Cao. 2020. MLGuard: Mitigating Poisoning Attacks in Privacy Preserving Distributed Collaborative Learning. In 2020 29th International Conference on Computer Communications and Networks (ICCCN).

[23]

Klaus Kursawe, George Danezis, and Markulf Kohlweiss. 2011. Privacy-Friendly Aggregation for the Smart-Grid. In Privacy Enhancing Technologies. Springer Berlin Heidelberg.

[24]

Changchang Liu, Supriyo Chakraborty, and Dinesh Verma. 2019. Secure Model Fusion for Distributed Learning Using Partial Homomorphic Encryption. Springer International Publishing.

[25]

E. Meijering. 2002. A chronology of interpolation: from ancient astronomy to modern signal and image processing. Proc. IEEE (2002).

[26]

Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. 2019. Exploiting Unintended Feature Leakage in Collaborative Learning. In 2019 IEEE Symposium on Security and Privacy (SP). 691–706. https://doi.org/10.1109/SP.2019.00029

[27]

Milad Nasr, Reza Shokri, and Amir Houmansadr. 2019. Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning. In 2019 IEEE Symposium on Security and Privacy (SP).

[28]

Thien Duc Nguyen, Phillip Rieger, Huili Chen, Hossein Yalame, Helen Möllering, Hossein Fereidooni, Samuel Marchal, Markus Miettinen, Azalia Mirhoseini, Shaza Zeitouni, Farinaz Koushanfar, Ahmad-Reza Sadeghi, and Thomas Schneider. 2022. FLAME: Taming Backdoors in Federated Learning. In 31st USENIX Security Symposium (USENIX Security 22). USENIX Association, Boston, MA, 1415–1432. https://www.usenix.org/conference/usenixsecurity22/presentation/nguyen

[29]

Takashi Nishide and Kouichi Sakurai. 2011. Distributed Paillier Cryptosystem without Trusted Dealer. In Information Security Applications, Yongwha Chung and Moti Yung (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 44–60.

[30]

Pascal Paillier. 1999. Public-Key Cryptosystems Based on Composite Degree Residuosity Classes. In Advances in Cryptology — EUROCRYPT ’99. Springer Berlin Heidelberg, Berlin, Heidelberg.

[31]

Le Trieu Phong, Yoshinori Aono, Takuya Hayashi, Lihua Wang, and Shiho Moriai. 2018. Privacy-Preserving Deep Learning via Additively Homomorphic Encryption. IEEE Transactions on Information Forensics and Security 13 (2018).

[32]

Tal Rabin. 1998. A Simplified Approach to Threshold and Proactive RSA. In Proceedings of the 18th Annual International Cryptology Conference on Advances in Cryptology(CRYPTO ’98). Springer-Verlag, Berlin, Heidelberg, 16 pages.

[33]

Adi Shamir. 1979. How to Share a Secret. Commun. ACM (1979).

[34]

Elaine Shi, T.-H Chan, Eleanor Rieffel, Richard Chow, and Dawn Song. 2011. Privacy-Preserving Aggregation of Time-Series Data. NDSS 2.

[35]

Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership Inference Attacks Against Machine Learning Models. In 2017 IEEE Symposium on Security and Privacy (SP).

[36]

Jinhyun So, Ramy E. Ali, Basak Guler, Jiantao Jiao, and Salman Avestimehr. 2021. Securing Secure Aggregation: Mitigating Multi-Round Privacy Leakage in Federated Learning. CoRR abs/2106.03328(2021).

[37]

Jinhyun So, Başak Göler, and A. Salman Avestimehr. 2021. Byzantine-Resilient Secure Federated Learning. IEEE Journal on Selected Areas in Communications 39 (2021).

[38]

Jinhyun So, Başak Güler, and A. Salman Avestimehr. 2021. Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning. IEEE Journal on Selected Areas in Information Theory 2 (2021).

[39]

Stacey Truex, Nathalie Baracaldo, Ali Anwar, Thomas Steinke, Heiko Ludwig, Rui Zhang, and Yi Zhou. 2019. A Hybrid Approach to Privacy-Preserving Federated Learning. In Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security(AISec’19). Association for Computing Machinery.

Digital Library

[40]

Thijs Veugen, Thomas Attema, and Gabriele Spini. 2019. An implementation of the Paillier crypto system with threshold decryption without a trusted dealer. Cryptology ePrint Archive, Report 2019/1136. https://ia.cr/2019/1136.

[41]

Danye Wu, Miao Pan, Zhiwei Xu, Yujun Zhang, and Zhu Han. 2020. Towards Efficient Secure Aggregation for Model Update in Federated Learning. In GLOBECOM 2020 - 2020 IEEE Global Communications Conference.

[42]

Guowen Xu, Hongwei Li, Sen Liu, Kan Yang, and Xiaodong Lin. 2020. VerifyNet: Secure and Verifiable Federated Learning. IEEE Transactions on Information Forensics and Security 15 (2020).

[43]

Runhua Xu, Nathalie Baracaldo, Yi Zhou, Ali Anwar, and Heiko Ludwig. 2019. HybridAlpha: An Efficient Approach for Privacy-Preserving Federated Learning. In Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security(AISec’19). Association for Computing Machinery.

Digital Library

[44]

Qian Yu, Netanel Raviv, Jinhyun So, and Amir Salman Avestimehr. 2018. Lagrange Coded Computing: Optimal Design for Resiliency, Security and Privacy. CoRR abs/1806.00939(2018). arXiv:1806.00939

[45]

Chengliang Zhang, Suyi Li, Junzhe Xia, Wei Wang, Feng Yan, and Yang Liu. 2020. BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association.

Cited By

Kim JLee S(2024)An Efficient Checkpoint Strategy for Federated Learning on Heterogeneous Fault-Prone NodesElectronics10.3390/electronics1306100713:6(1007)Online publication date: 7-Mar-2024
https://doi.org/10.3390/electronics13061007
Taiello RÖnen MGritti CLorenzi M(2024)Let Them Drop: Scalable and Efficient Federated Learning Solutions Agnostic to StragglersProceedings of the 19th International Conference on Availability, Reliability and Security10.1145/3664476.3664488(1-12)Online publication date: 30-Jul-2024
https://dl.acm.org/doi/10.1145/3664476.3664488
Feng XWang XLiu HYang HWang L(2024)A Privacy-Preserving Aggregation Scheme With Continuous Authentication for Federated Learning in VANETsIEEE Transactions on Vehicular Technology10.1109/TVT.2024.336994273:7(9465-9477)Online publication date: Jul-2024
https://doi.org/10.1109/TVT.2024.3369942
Show More Cited By

Index Terms

Learning from Failures: Secure and Fault-Tolerant Aggregation for Federated Learning
1. Security and privacy
  1. Network security
    1. Security protocols
  2. Security services
    1. Privacy-preserving protocols

Recommendations

Secure and efficient multi-key aggregation for federated learning
Abstract
Federated learning (FL) is a distributed machine learning framework that aims to provide privacy for local datasets while learning a global machine learning model. However, the updates exchanged in FL may indirectly reveal information about the ...
Verifiable Secure Aggregation Protocol Under Federated Learning
Artificial Intelligence Security and Privacy
Abstract
Federated learning is a new machine learning paradigm used for collaborative training models among multiple devices. In federated learning, multiple clients participate in model training locally and use decentralized learning methods to ensure the ...
Post-quantum Privacy-Preserving Aggregation in Federated Learning Based on Lattice
Cyberspace Safety and Security
Abstract
With the fast growth of quantum computers in recent years, the use of post-quantum cryptography has steadily become vital. In the privacy-preserving aggregation of federated learning, different cryptographic primitives, such as encryption, key ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

ACSAC '22: Proceedings of the 38th Annual Computer Security Applications Conference

December 2022

1021 pages

ISBN:9781450397599

DOI:10.1145/3564625

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 December 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Artifacts Evaluated & Functional / v1.1

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

3IA Côte dðAzur Program
H2020 COLLABS

Conference

ACSAC

ACSAC: Annual Computer Security Applications Conference

December 5 - 9, 2022

TX, Austin, USA

Acceptance Rates

Overall Acceptance Rate 104 of 497 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
412
Total Downloads

Downloads (Last 12 months)158
Downloads (Last 6 weeks)15

Reflects downloads up to 04 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Kim JLee S(2024)An Efficient Checkpoint Strategy for Federated Learning on Heterogeneous Fault-Prone NodesElectronics10.3390/electronics1306100713:6(1007)Online publication date: 7-Mar-2024
https://doi.org/10.3390/electronics13061007
Taiello RÖnen MGritti CLorenzi M(2024)Let Them Drop: Scalable and Efficient Federated Learning Solutions Agnostic to StragglersProceedings of the 19th International Conference on Availability, Reliability and Security10.1145/3664476.3664488(1-12)Online publication date: 30-Jul-2024
https://dl.acm.org/doi/10.1145/3664476.3664488
Feng XWang XLiu HYang HWang L(2024)A Privacy-Preserving Aggregation Scheme With Continuous Authentication for Federated Learning in VANETsIEEE Transactions on Vehicular Technology10.1109/TVT.2024.336994273:7(9465-9477)Online publication date: Jul-2024
https://doi.org/10.1109/TVT.2024.3369942
Fan JWu KTang GZhou YHuang S(2024)Taking Advantage of the Mistakes: Rethinking Clustered Federated Learning for IoT Anomaly DetectionIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2024.337990535:6(862-876)Online publication date: Jun-2024
https://doi.org/10.1109/TPDS.2024.3379905
Feng XLiu HYang HXie QWang L(2024)Batch-Aggregate: Efficient Aggregation for Private Federated Learning in VANETsIEEE Transactions on Dependable and Secure Computing10.1109/TDSC.2024.3364371(1-15)Online publication date: 2024
https://doi.org/10.1109/TDSC.2024.3364371
Ming YWang SWang CLiu HDeng YZhao YFeng J(2024)VCSA: Verifiable and collusion-resistant secure aggregation for federated learning using symmetric homomorphic encryptionJournal of Systems Architecture10.1016/j.sysarc.2024.103279156(103279)Online publication date: Nov-2024
https://doi.org/10.1016/j.sysarc.2024.103279
Karakoç FKüpçü AÖnen M(2024)Fault Tolerant and Malicious Secure Federated LearningCryptology and Network Security10.1007/978-981-97-8016-7_4(73-95)Online publication date: 29-Sep-2024
https://doi.org/10.1007/978-981-97-8016-7_4
Onsu MKantarci BBoukerche A(2023)How to cope with malicious federated learning clientsComputer Networks: The International Journal of Computer and Telecommunications Networking10.1016/j.comnet.2023.109938234:COnline publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1016/j.comnet.2023.109938

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents