More Web Proxy on the site http://driver.im/

research-article

FedShare: Secure Aggregation based on Additive Secret Sharing in Federated Learning

Authors:

Hamid Fazli Khojir,

Noman MohammedAuthors Info & Claims

IDEAS '23: Proceedings of the 27th International Database Engineered Applications Symposium

Pages 25 - 33

https://doi.org/10.1145/3589462.3589504

Published: 26 May 2023 Publication History

Abstract

Federated learning is a machine learning technique where multiple clients with local data collaborate in training a machine learning model. In FedAvg, the main federated learning algorithm, clients train machine learning models locally and share the trained model with the server. While the sensitive data will never be sent to the server, a malicious server can construct the original training data by having access to the clients’ models in each training round. Secure aggregation techniques such as cryptography, trusted execution environment, or differential privacy are used to solve this problem. However, these techniques incur computation and communication overhead or affect the model’s accuracy. In this paper, we consider a secure multi-party computation setup where clients use additive secret sharing to send their models to multiple servers. Our solution provides secure aggregation as long as there are at least two non-colluding servers. Moreover, we provide mathematical proof to show that the securely aggregated model at the end of each training round is exactly equal to the one provided by FedAvg without affecting accuracy and with efficient communication and computation. In comparison with SCOTCH, the state-of-the-art secure aggregation solution, experimental results show that our approach is 557% faster compared to SCOTCH and at the same time it reduces the communication cost of clients by 25%. Additionally, the accuracy of the trained model is exactly as FedAvg under balanced, unbalanced, IID, and Non-IID data distributions while it is only 8% slower.

References

[1]

Naman Agarwal, Ananda Theertha Suresh, Felix Xinnan X Yu, Sanjiv Kumar, and Brendan McMahan. 2018. cpSGD: Communication-efficient and differentially-private distributed SGD. Advances in Neural Information Processing Systems 31 (2018).

[2]

Marzia Ahmed. 2020. Maternal Health Risk Data Set Data Set. https://archive.ics.uci.edu/ml/datasets/Maternal+Health+Risk+Data+Set

[3]

Constance Beguier, Mathieu Andreux, and Eric W Tramel. 2020. Efficient sparse secure aggregation for federated learning. arXiv preprint arXiv:2007.14861 (2020).

[4]

Amos Beimel. 2011. Secret-sharing schemes: A survey. In International conference on coding and cryptology. Springer, 11–46.

[5]

James Henry Bell, Kallista A. Bonawitz, Adrià Gascón, Tancrède Lepoint, and Mariana Raykova. 2020. Secure Single-Server Aggregation with (Poly)Logarithmic Overhead. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security (Virtual Event, USA) (CCS ’20). Association for Computing Machinery, New York, NY, USA, 1253–1269. https://doi.org/10.1145/3372297.3417885

Digital Library

[6]

Abhishek Bhowmick, John Duchi, Julien Freudiger, Gaurav Kapoor, and Ryan Rogers. 2018. Protection against reconstruction and its applications in private federated learning. arXiv preprint arXiv:1812.00984 (2018).

[7]

Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. 2017. Practical Secure Aggregation for Privacy-Preserving Machine Learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (Dallas, Texas, USA) (CCS ’17). Association for Computing Machinery, New York, NY, USA, 1175–1191. https://doi.org/10.1145/3133956.3133982

Digital Library

[8]

Chen Chen, Lingjuan Lyu, Han Yu, and Gang Chen. 2022. Practical Attribute Reconstruction Attack Against Federated Learning. IEEE Transactions on Big Data (2022).

[9]

Computer Science at Columbia University. 2022. COMS W4261:Introduction to Cryptography. https://www1.cs.columbia.edu/ tal/4261/F19/secretsharingf19.pdf.

[10]

Mahir Can Doganay, Thomas B Pedersen, Yücel Saygin, Erkay Savaş, and Albert Levi. 2008. Distributed privacy preserving k-means clustering with additive secret sharing. In Proceedings of the 2008 international workshop on Privacy and anonymity in information society. 3–11.

Digital Library

[11]

European Commission. 2022. Rules for business and organisations. https://ec.europa.eu/info/law/law-topic/data-protection/reform/rules-business-and-organisations_en.

[12]

David Evans, Vladimir Kolesnikov, Mike Rosulek, 2018. A pragmatic introduction to secure multi-party computation. Foundations and Trends® in Privacy and Security 2, 2-3 (2018), 70–246.

[13]

Hossein Fereidooni, Samuel Marchal, Markus Miettinen, Azalia Mirhoseini, Helen Möllering, Thien Duc Nguyen, Phillip Rieger, Ahmad-Reza Sadeghi, Thomas Schneider, Hossein Yalame, 2021. SAFELearn: secure aggregation for private federated learning. In 2021 IEEE Security and Privacy Workshops (SPW). IEEE, 56–62.

[14]

Karan Ganju, Qi Wang, Wei Yang, Carl A Gunter, and Nikita Borisov. 2018. Property inference attacks on fully connected neural networks using permutation invariant representations. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security. 619–633.

Digital Library

[15]

Robin C. Geyer, Tassilo Klein, and Moin Nabi. 2017. Differentially Private Federated Learning: A Client Level Perspective. CoRR abs/1712.07557 (2017). arXiv:1712.07557http://arxiv.org/abs/1712.07557

[16]

Slawomir Goryczka and Li Xiong. 2015. A comprehensive comparison of multiparty secure additions with differential privacy. IEEE transactions on dependable and secure computing 14, 5 (2015), 463–477.

[17]

HHS.gov. 2022. The HIPAA Privacy Rule. https://www.hhs.gov/hipaa/for-professionals/privacy/index.html.

[18]

Briland Hitaj, Giuseppe Ateniese, and Fernando Perez-Cruz. 2017. Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (Dallas, Texas, USA) (CCS ’17). Association for Computing Machinery, New York, NY, USA, 603–618. https://doi.org/10.1145/3133956.3134012

Digital Library

[19]

Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, Philip S Yu, and Xuyun Zhang. 2021. Membership inference attacks on machine learning: A survey. ACM Computing Surveys (CSUR) (2021).

[20]

Hongsheng Hu, Zoran Salcic, Lichao Sun, Gillian Dobbie, and Xuyun Zhang. 2021. Source inference attacks in federated learning. In 2021 IEEE International Conference on Data Mining (ICDM). IEEE, 1102–1107.

[21]

Swanand Kadhe, Nived Rajaraman, O Ozan Koyluoglu, and Kannan Ramchandran. 2020. Fastsecagg: Scalable secure aggregation for privacy-preserving federated learning. arXiv preprint arXiv:2009.11248 (2020).

[22]

Qiongxiu Li, Ignacio Cascudo, and Mads Græsbøll Christensen. 2019. Privacy-preserving distributed average consensus based on additive secret sharing. In 2019 27th European Signal Processing Conference (EUSIPCO). IEEE, 1–5.

[23]

Mohammad Malekzadeh, Burak Hasircioglu, Nitish Mital, Kunal Katarya, Mehmet Emre Ozfatura, and Deniz Gündüz. 2021. Dopamine: Differentially private federated learning on medical data. arXiv preprint arXiv:2101.11693 (2021).

[24]

Kalikinkar Mandal and Guang Gong. 2019. PrivFL: Practical privacy-preserving federated regressions on high-dimensional data over mobile networks. In Proceedings of the 2019 ACM SIGSAC Conference on Cloud Computing Security Workshop. 57–68.

Digital Library

[25]

H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. 2016. Communication-Efficient Learning of Deep Networks from Decentralized Data. (2016). https://doi.org/10.48550/ARXIV.1602.05629

[26]

H Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang. 2017. Learning differentially private recurrent language models. arXiv preprint arXiv:1710.06963 (2017).

[27]

Payman Mohassel and Yupeng Zhang. 2017. Secureml: A system for scalable privacy-preserving machine learning. In 2017 IEEE symposium on security and privacy (SP). IEEE, 19–38.

[28]

Arup Mondal, Yash More, Prashanthi Ramachandran, Priyam Panda, Harpreet Virk, and Debayan Gupta. 2022. Scotch: an efficient secure computation framework for secure aggregation. arXiv preprint arXiv:2201.07730 (2022).

[29]

Milad Nasr, Reza Shokri, and Amir Houmansadr. 2019. Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning. In 2019 IEEE Symposium on Security and Privacy (SP). IEEE. https://doi.org/10.1109/sp.2019.00065

[30]

John Nguyen, Kshitiz Malik, Hongyuan Zhan, Ashkan Yousefpour, Mike Rabbat, Mani Malek, and Dzmitry Huba. 2022. Federated learning with buffered asynchronous aggregation. In International Conference on Artificial Intelligence and Statistics. PMLR, 3581–3607.

[31]

M Sadegh Riazi, Christian Weinert, Oleksandr Tkachenko, Ebrahim M Songhori, Thomas Schneider, and Farinaz Koushanfar. 2018. Chameleon: A hybrid secure computation framework for machine learning applications. In Proceedings of the 2018 on Asia conference on computer and communications security. 707–721.

Digital Library

[32]

Ahmed Salem, Apratim Bhattacharya, Michael Backes, Mario Fritz, and Yang Zhang. 2020. { Updates-Leak} : Data Set Inference and Reconstruction Attacks in Online Learning. In 29th USENIX security symposium (USENIX Security 20). 1291–1308.

[33]

Sinem Sav, Apostolos Pyrgelis, Juan R Troncoso-Pastoriza, David Froelicher, Jean-Philippe Bossuat, Joao Sa Sousa, and Jean-Pierre Hubaux. 2020. POSEIDON: Privacy-preserving federated neural network learning. arXiv preprint arXiv:2009.00349 (2020).

[34]

Adi Shamir. 1979. How to share a secret. Commun. ACM 22, 11 (1979), 612–613.

Digital Library

[35]

Reza Shokri and Vitaly Shmatikov. 2015. Privacy-Preserving Deep Learning. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (Denver, Colorado, USA) (CCS ’15). Association for Computing Machinery, New York, NY, USA, 1310–1321. https://doi.org/10.1145/2810103.2813687

Digital Library

[36]

Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP). IEEE, 3–18.

[37]

Runhua Xu, Nathalie Baracaldo, Yi Zhou, Ali Anwar, and Heiko Ludwig. 2019. HybridAlpha. In Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security - AISec'19. ACM Press. https://doi.org/10.1145/3338501.3357371

Digital Library

[38]

Chengliang Zhang, Suyi Li, Junzhe Xia, Wei Wang, Feng Yan, and Yang Liu. 2020. BatchCrypt: Efficient Homomorphic Encryption for Cross-Silo Federated Learning. In 2020 USENIX Annual Technical Conference (USENIX ATC 20). USENIX Association, 493–506. https://www.usenix.org/conference/atc20/presentation/zhang-chengliang

[39]

Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep Leakage from Gradients. https://doi.org/10.48550/ARXIV.1906.08935

Cited By

Ting YChi PKuo C(2024)A Reliable Aggregation Method Based on Threshold Additive Secret Sharing in Federated Learning with Quality of Service (QoS) SupportApplied Sciences10.3390/app1419895914:19(8959)Online publication date: 4-Oct-2024
https://doi.org/10.3390/app14198959
Asare BBranco PKiringa IYeap T(2024)AddShare+: Efficient Selective Additive Secret Sharing Approach for Private Federated Learning2024 IEEE 11th International Conference on Data Science and Advanced Analytics (DSAA)10.1109/DSAA61799.2024.10722785(1-10)Online publication date: 6-Oct-2024
https://doi.org/10.1109/DSAA61799.2024.10722785
Wang DChen LLu XWang YShen YXu J(2024)FedDBO: A Novel Federated Learning Approach for Communication Cost and Data Heterogeneity Using Dung Beetle OptimizerIEEE Access10.1109/ACCESS.2024.337927312(43396-43409)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3379273
Show More Cited By

Index Terms

FedShare: Secure Aggregation based on Additive Secret Sharing in Federated Learning
1. Security and privacy
  1. Human and societal aspects of security and privacy
    1. Economics of security and privacy
  2. Security services
    1. Privacy-preserving protocols

Recommendations

Secure Single-Server Aggregation with (Poly)Logarithmic Overhead
CCS '20: Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security

Secure aggregation is a cryptographic primitive that enables a server to learn the sum of the vector inputs of many clients. Bonawitz et al. (CCS 2017) presented a construction that incurs computation and communication for each client linear in the ...
Verifiable Secure Aggregation Protocol Under Federated Learning
Artificial Intelligence Security and Privacy
Abstract
Federated learning is a new machine learning paradigm used for collaborative training models among multiple devices. In federated learning, multiple clients participate in model training locally and use decentralized learning methods to ensure the ...
DealSecAgg: Efficient Dealer-Assisted Secure Aggregation for Federated Learning
ARES '24: Proceedings of the 19th International Conference on Availability, Reliability and Security

Federated learning eliminates the necessity of transferring private training data and instead relies on the aggregation of model updates. Several publications on privacy attacks show how these individual model updates are vulnerable to the extraction of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

IDEAS '23: Proceedings of the 27th International Database Engineered Applications Symposium

May 2023

222 pages

ISBN:9798400707445

DOI:10.1145/3589462

Editors:
Richard Chbeir
University Pau & Pays Adour, France
,
Mirjana Ivanovic
University of Novi Sad, Serbia
,
Yannis Manolopoulos
Open University of Cyprus, Cyprus
,
Peter Z. Revesz
University of Nebraska Lincoln, USA

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 May 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Natural Sciences and Engineering Research Council of Canada (NSERC) Discovery Grant

Conference

IDEAS '23

IDEAS '23: International Database Engineered Applications Symposium Conference

May 5 - 7, 2023

Heraklion, Crete, Greece

Acceptance Rates

Overall Acceptance Rate 74 of 210 submissions, 35%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
387
Total Downloads

Downloads (Last 12 months)252
Downloads (Last 6 weeks)30

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Ting YChi PKuo C(2024)A Reliable Aggregation Method Based on Threshold Additive Secret Sharing in Federated Learning with Quality of Service (QoS) SupportApplied Sciences10.3390/app1419895914:19(8959)Online publication date: 4-Oct-2024
https://doi.org/10.3390/app14198959
Asare BBranco PKiringa IYeap T(2024)AddShare+: Efficient Selective Additive Secret Sharing Approach for Private Federated Learning2024 IEEE 11th International Conference on Data Science and Advanced Analytics (DSAA)10.1109/DSAA61799.2024.10722785(1-10)Online publication date: 6-Oct-2024
https://doi.org/10.1109/DSAA61799.2024.10722785
Wang DChen LLu XWang YShen YXu J(2024)FedDBO: A Novel Federated Learning Approach for Communication Cost and Data Heterogeneity Using Dung Beetle OptimizerIEEE Access10.1109/ACCESS.2024.337927312(43396-43409)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3379273
Izadi SAhmadi M(2024)New fusion loss function based on knowledge generation using Gumbel-SoftMax for federated learningThe Journal of Supercomputing10.1007/s11227-024-06593-w81:1Online publication date: 19-Oct-2024
https://doi.org/10.1007/s11227-024-06593-w
Chen CWang KLi PSakurai K(2024)Enhancing Security and Efficiency: A Lightweight Federated Learning ApproachAdvanced Information Networking and Applications10.1007/978-3-031-57916-5_30(349-359)Online publication date: 9-Apr-2024
https://doi.org/10.1007/978-3-031-57916-5_30

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents