Masking and Homomorphic Encryption-Combined Secure Aggregation for Privacy-Preserving Federated Learning
Abstract
:1. Introduction
- Hybrid Secure Aggregation Strategy: We introduce a novel and practical secure aggregation strategy that integrates the advantages of both masking-based and HE-based aggregation methods. Compared to traditional masking-based techniques that often require complex collaborative mask management, our method eliminates the operational overhead associated with mask coordination and employs a new MKHE technique to encrypt masks. The encrypted masks are efficiently removed from the aggregated result using a homomorphic additive operation. This eliminates the need for collaborative operations while ensuring robust privacy for local model parameters.
- Automatic Decryption and Efficient Global Model Update: The proposed MKHE technique allows the use of different keys and facilitates automatic decryption of the aggregated ciphertexts. Consequently, the server can directly retrieve the actual global model update without requiring an additional decryption process, a limitation seen in conventional MKHE models. Thus, our approach minimizes communication rounds, requiring only two interactions to complete a global model update under no-dropout conditions.
- Key Management Independence: In our model, nodes can manage their keys independently of the aggregation group’s composition. In the existing MKHE-based aggregation techniques, where local parameters are directly encrypted, nodes need to update their keys whenever the aggregation group changes. This introduces considerable computational and operational overhead due to the need for the collaborative key updates among the newly formed group. In contrast, our method ensures that each node’s keys remain unchanged throughout the entire federated learning process. This is because only the mask value, not the local parameters, is encrypted, and individual mask values are independently refreshed by each node in every round of federated learning. Even in the presence of dropouts, nodes can perform the re-aggregation using their existing keys, eliminating the need for additional key update protocols.
2. Related Works
3. MHESA: Masking and Homomorphic Encryption-Combined Secure Aggregation
3.1. Background and System Overview
3.1.1. Federated Learning Model
3.1.2. RLWE-Based HE Scheme
- KeyGen(n, h, q, p, σ) samples s ← HWT(h), A ← and e ← D(σ2). It sets the secret key as (1, s) and the public key pk as (B, A) in where B = −A·s + e (mod qL).
- Ecd(z; ) generates an integral plaintext polynomial m(X) for a given (n/2)-dimensional vector . It calculates , where represents a scaling factor, and is a natural projection defined by for a multiplicative subgroup T of satisfying . is a canonical embedding map from integral polynomial to elements in the complex field . It computes a polynomial whose evaluation values at the roots, the complex primitive roots of unity in the extension field , correspond to the given vector of complex numbers.
- Dcd(m; ) returns the vector for an input polynomial m in R, i.e., for .
- Enc(m, pk), for a public key pk and a plaintext polynomial m, samples v ← ZO(0.5), and e0, e1 ← D(σ2), and outputs a ciphertext c = (c0, c1), where c0 = v·A + e0 (mod qL) and c1 = v·B + m + e1 (mod qL).
- Dec(c, sk), for a ciphertext c = (c0, c1) and a secret key sk, outputs c1 + c0·s mod ql.
3.1.3. System Overview
- KeyGen(n, h, q, p, σ, A, Q) generates the public-private key pair <pki, ski> and the commitment ci of each node ui. It samples si ← HWT(h), vi ← ZO(0.5) and ei, e0i ← D(σ2). It sets the secret key as ski = (1, si) and public key pki = −A·ski + ei (mod Q). Then, it sets a commitment ci for vi such as ci = A·vi + e0i (mod Q).
- GroupKeyGen(PK, C, UT) generates a group public key PKT and a group commitment CT for a given node set , where PK = {pki} and C = {ci} for all ui in U. It sets PKT as and CT as .
- Enc(mi, PKT, CT, ski) outputs a ciphertext Ei for a plaintext polynomial mi. For given group public key PKT and group commitment CT, it samples e1i ← D(σ2), and outputs a ciphertext (mod Q).
- Dec(ET, UT) adds all ciphertexts Ei in ET, where ET = {Ei} for all ui in UT. If ciphertexts obtained from all nodes in UT are added, it outputs the sum of plaintext polynomials generated by all nodes in UT such as .
- (1)
- Privacy of local datasets and model parameters: All data stored on each node’s local device and the local model parameters transmitted over the network must remain confidential, shielded not only from other nodes but also from FS. FS only has access to the aggregated sum of all local updates provided by the nodes.
- (2)
- Robustness to dropouts: In scenarios where data transmission is disrupted due to network issues or device malfunctions, FS must still be able to accurately compute a global model update.
3.2. MHESA Protocol
- FS sets n, h, q and σ as described in Section 3.1.2 and generates Q = q2 and a public ring element A ← with the vector of coefficients [a0, …, an−1] in . FS publishes <n, h, q, σ, A, Q> to all nodes.
- Each ui ∈ U generates its key pair <pki, ski> and a commitment ci by KeyGen(n, h, q, σ, A, Q), where ski = (1, si), pki = −A·ski + ei (mod Q) and ci = A·vi + e0i (mod Q). Then, ui responds to FS with <pki, ci, xi>, where xi represents the size of ui’s local dataset.
- FS sets PK = {pki}, C = {ci} and X = {xi} for all ui in U.
- FS sends a start message to all nodes.
- All available nodes respond to FS with their xis, where xi represents the size of ui’s local dataset.
- FS sets UT as the t-th node group for all replied nodes and generates the t-th group parameters PKT and CT for all nodes in UT by GroupKeyGen(PK, C, UT), where and . It also sets the total size of datasets as . Then, it broadcasts <PKT, CT, XT> to all nodes in UT.
- 4.
- For each ui in UT, let represent a set of local model parameters of ui at the t-th iteration. ui selects a random real number and generates a masked local update according to the following Equation (5):Next, ui generates a plaintext polynomial and calculates the encryption of , , as shown in Equation (6).Note that includes both the encryption of using the group public key PKT and the partial decryption by ui. ui sends <, > to FS.
- 5.
- FS calculates and (mod q). Here, . FS computes . Finally, FS updates the t-th global update wt with the average of all local updates as described in Equation (7):FS distributes to all nodes in U.
- 6.
- Each ui updates its local model with .
3.3. Dropout Management
4. Analysis
4.1. Correctness and Security
1: | (mod q) | |
2: | (mod q) | |
3: | (mod q) | |
4: | (mod q) | |
5: | (mod q) | |
6: | (mod q) | |
7: | (mod q) | |
8: | (mod q) | |
9: | (mod q) | |
10: | (mod q) | |
11: | where in CKKS-HE | (mod q) |
4.2. Simulated Performance
4.2.1. Experimental Environment
4.2.2. Experimental Results
5. Discussion and Conclusions
Author Contributions
Funding
Data Availability Statement
Conflicts of Interest
References
- McMahan, H.B.; Moore, E.; Ramage, D.; Hampson, S.; Arcas, B.A. Communication-Efficient Learning of Deep Networks from Decentralized Data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), Fort Lauderdale, FL, USA, 9–11 May 2017; Volume 54. [Google Scholar]
- Zhu, L.; Liu, Z.; Han, S. Deep leakage from gradients. arXiv 2019, arXiv:1906.08935. [Google Scholar]
- Wang, Z.; Song, M.; Zhang, Z.; Song, Y.; Wang, Q.; Qi, H. Beyond inferring class representatives: User-level privacy leakage from federated learning. In Proceedings of the IEEE INFOCOM, Paris, France, 29 April–2 May 2019; pp. 2512–2520. [Google Scholar]
- Geiping, J.; Bauermeister, H.; Dröge, H.; Moeller, M. Inverting gradients—How easy is it to break privacy in federated learning? In Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020), Online, 6–12 December 2020.
- Yao, A.C. Protocols for secure computations. In Proceedings of the 23rd IEEE Annual Symposium on Foundations of Computer Sciecne (sfcs 1982), Chicago, IL, USA, 3–5 November 1982; pp. 160–164. [Google Scholar]
- Shamir, A. How to share a secret. Commun. ACM 1979, 22, 612–613. [Google Scholar] [CrossRef]
- Geyer, R.C.; Klein, T.; Nabi, M. Differentially private federated learning: A client level perspective. In Proceedings of the NIPS 2017 Workshop: Machine Learning on the Phone and other Consumer Devices, Long Beach, CA, USA, 8 December 2017. [Google Scholar]
- Wei, K.; Li, J.; Ding, M.; Ma, C.; Yang, H.H.; Farokhi, F.; Jin, S.; Quek, T.Q.S.; Poor, H.V. Federated Learning with Differential Privacy: Algorithms and Performance Analysis. IEEE Trans. Inf. Forensics Secur. 2020, 15, 3454–3469. [Google Scholar] [CrossRef]
- Leontiadis, I.; Elkhiyaoui, K.; Molva, R. Private and dynamic timeseries data aggregation with trust relaxation. In Proceedings of the International Conferences on Cryptology and Network Security (CANS 2014), Seoul, Republic of Korea, 1–3 December 2010; Springer: Berlin/Heidelberg, Germany, 2014; pp. 305–320. [Google Scholar]
- Rastogi, V.; Nath, S. Differentially private aggregation of distributed time-series with transformation and encryption. In Proceedings of the ACM SIGMOD International Conference on Management of Data (SIGMOD 10), Indianapolis, IN, USA, 6–10 June 2010; pp. 735–746. [Google Scholar]
- Halevi, S.; Lindell, Y.; Pinkas, B. Secure computation on the Web: Computing without simultaneous interaction. In Advances in Cryptology—CRYPTO 2011; Springer: Berlin/Heidelberg, Germany, 2011; pp. 132–150. [Google Scholar]
- Leontiadis, I.; Elkhiyaoui, K.; Önen, M.; Molva, R. PUDA—Privacy and Unforgeability for Data Aggregation. In Cryptology and Network Security. CANS 2015; Springer: Cham, Switzerland, 2015; pp. 3–18. [Google Scholar]
- Aono, Y.; Hayashi, T.; Wang, L.; Moriai, S. Privacy-preserving deep learning via additively homomorphic encryption. IEEE Trans. Inf. Forensics Secur. 2017, 13, 1333–1345. [Google Scholar]
- Fang, H.; Qian, Q. Privacy Preserving Machine Learning with Homomorphic Encryption and Federated Learning. Future Internet 2021, 13, 94. [Google Scholar] [CrossRef]
- Park, J.; Lim, H. Privacy-Preserving Federated Learning Using Homomorphic Encryption. Appl. Sci. 2022, 12, 734. [Google Scholar] [CrossRef]
- Liu, W.; Zhou, T.; Chen, L.; Yang, H.; Han, J.; Yang, X. Round efficient privacy-preserving federated learning based on MKFHE. Comput. Stand. Interfaces 2024, 87, 103773. [Google Scholar] [CrossRef]
- Jin, W.; Yao, Y.; Han, S.; Gu, J.; Joe-Wong, C.; Ravi, S.; Avestimehr, S.; He, C. FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning System. arXiv 2023, arXiv:2303.10837. [Google Scholar]
- Wibawa, F.; Catak, F.O.; Kuzlu, M.; Sarp, S.; Cali, U. Homomorphic Encryption and Federated Learning based Privacy-Preserving CNN Training: COVID-19 Detection Use-Case. In Proceedings of the 2022 European Interdisciplinary Cybersecurity Conference (EICC’22), Barcelona, Spain, 15–16 June 2022; Association for Computing Machinery: New York, NY, USA, 2022; pp. 85–90. [Google Scholar] [CrossRef]
- Hijazi, N.M.; Aloqaily, M.; Guizani, M.; Ouni, B.; Karray, F. Secure Federated Learning With Fully Homomorphic Encryption for IoT Communications. IEEE Internet Things J. 2024, 11, 4289–4300. [Google Scholar] [CrossRef]
- Sanon, S.P.; Reddy, R.; Lipps, C.; Schotten, H.D. Secure Federated Learning: An Evaluation of Homomorphic Encrypted Network Traffic Prediction. In Proceedings of the IEEE 20th Consumer Communications & Networking Conference (CCNC), Las Vegas, NV, USA, 8–11 January 2023; pp. 1–6. [Google Scholar] [CrossRef]
- Cheon, J.H.; Kim, A.; Kim, M.; Song, Y. Homomorphic Encryption for Arithmetic of Approximate Numbers. In Advances in Cryptology-ASIASCRYPT 2017; Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2017; Volume 10624. [Google Scholar] [CrossRef]
- Bonawitz, K.; Ivanov, V.; Kreuter, B.; Marcedoney, A.; McMahan, H.; Patel, S.; Ramage, D.; Segal, A.; Seth, K. Practical Secure Aggregation for Federated Learning on User-Held Data. arXiv 2016, arXiv:1611.04482. [Google Scholar]
- Bonawitz, K.; Ivanov, V.; Kreuter, B.; Marcedone, A.; McMahan, H.B.; Patel, S.; Ramage, D.; Segal, A.; Seth, K. Practical secure aggregation for privacy-preserving machine learning. In Proceedings of the ACM SIGSAC Conferences on Computer and Communications Security, Dallas, TX, USA, 30 October–3 November 2017; pp. 1175–1191. [Google Scholar]
- Ács, G.; Castelluccia, C. I have a DREAM! (DiffeRentially privatE smArt Metering). In Information Hiding. IH 2011; Springer: Berlin/Heidelberg, Germany, 2011; pp. 118–132. [Google Scholar]
- Goryczka, S.; Xiong, L. A comprehensive comparison of multiparty secure additions with differential privacy. IEEE Trans. Dependable Secur. Comput. 2017, 14, 463–477. [Google Scholar] [CrossRef] [PubMed]
- Elahi, T.; Danezis, G.; Goldberg, I. Privex: Private collection of traffic statistics for anonymous communication networks. In Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security, Scottsdale, AZ, USA, 3–7 November 2014; pp. 1068–1079. [Google Scholar]
- Jansen, R.; Johnson, A. Safely Measuring Tor. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, Vienna, Austria, 24–28 October 2016; pp. 1553–1567. [Google Scholar]
- So, J.; Güler, B.; Avestimehr, A.S. Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning. arXiv 2020, arXiv:2002.04156. [Google Scholar] [CrossRef]
- Kim, J.; Park, G.; Kim, M.; Park, S. Cluster-Based Secure Aggregation for Federated Learning. Electronics 2023, 12, 870. [Google Scholar] [CrossRef]
- LeCun, Y.; Cortes, C.; Burges, C.J. MNIST Handwritten Digit Database. 2010. Available online: http://yann.lecun.com/exdb/mnist (accessed on 4 October 2024).
Number of Nodes | Data Size per Node | ||
---|---|---|---|
IID Distribution | Non-IID Distribution | ||
Minimum | Maximum | ||
1 | 60,000 | 60,000 | |
10 | 6000 | 500 | 12,700 |
20 | 3000 | 400 | 4800 |
50 | 1200 | 100 | 2200 |
100 | 600 | 100 | 1200 |
Parameters | Values |
---|---|
The number of nodes | 1, 10, 20, 50, 100 |
The number of rounds for federated learning | 100 |
The number of weight parameters | 21,840 |
The size of n as the degree of ring polynomial (bits) | 16 |
The size of q as the modulus for decryption (bits) | 800 |
Size of Q as the modulus for encryption (bits) | 1600 |
Round | Accuracy of the Proposed Model (N = 10, IID) (%) | Accuracy of Raw Data Aggregation (%) |
---|---|---|
10 | 91.28 (0.10) * | 91.27 (0.11) * |
20 | 94.33 (0.08) * | 94.32 (0.07) * |
40 | 96.36 (0.05) * | 96.36 (0.05) * |
60 | 97.08 (0.05) * | 97.09 (0.05) * |
80 | 97.42 (0.05) * | 97.43 (0.04) * |
100 | 97.75 (0.04) * | 97.74 (0.05) * |
Average Accuracy (%) | |||||||||
---|---|---|---|---|---|---|---|---|---|
N | N = 1 | N = 10 | N = 20 | N = 50 | N = 100 | ||||
Round | IID | Non-IID | IID | Non-IID | IID | Non-IID | IID | Non-IID | |
10 | 97.69 | 91.27 | 87.96 | 84.29 | 87.47 | 53.65 | 66.98 | 16.26 | 35.55 |
(0.07) * | (0.19) * | (2.55) * | (0.15) * | (0.17) * | (0.46) * | (2.05) * | (0.2) * | (3.67) * | |
20 | 98.35 | 94.27 | 92.15 | 91.19 | 91.19 | 76.70 | 86.15 | 47.21 | 66.09 |
(0.08) * | (0.03) * | (3.03) * | (0.07) * | (0.18) * | (0.03) * | (0.32) * | (0.76) * | (1.62) * | |
40 | 98.78 | 96.33 | 94.72 | 94.27 | 95.05 | 89.59 | 91.27 | 71.77 | 84.70 |
(0.06) * | (0.04) * | (2.11) * | (0.04) * | (0.12) * | (0.12) * | (0.16) * | (0.34) * | (0.43) * | |
60 | 98.94 | 97.08 | 95.68 | 95.54 | 96.21 | 92.10 | 93.06 | 85.77 | 88.9 |
(0.02) * | (0.04) * | (1.81) * | (0.02) * | (0.06) * | (0.06) * | (0.14) * | (0.12) * | (0.19) * | |
80 | 99.02 | 97.53 | 96.28 | 96.31 | 96.67 | 93.35 | 94.13 | 88.65 | 90.74 |
(0.07) * | (0.05) * | (1.56) * | (0.03) * | (0.1) * | (0.04) * | (0.20) * | (0.05) * | (0.12) * | |
100 | 99.16 | 97.74 | 96.7 | 96.73 | 97.05 | 94.13 | 94.93 | 90.39 | 91.79 |
(0.05) * | (0.05) * | (1.32) * | (0.04) * | (0.1) * | (0.03) * | (0.22) * | (0.03) * | (0.15) * |
Parameters | Size (Byte) |
---|---|
A | 13,107,200 (=12.5 MB) |
pki, PKU | 13,631,488 (=13 MB) |
ci, CU | 6,815,744 (=6.5 MB) |
wi, Di, DU, w | 174,720 (=0.17 MB) |
Ei, EU | 13,107,200 (=12.5 MB) |
Phase | Actor | Operations | Average Time (ms) |
---|---|---|---|
Setup | Server | Creating a ring polynomial A | 34.9399 |
Sending A to each node | 1568.971 | ||
Node | Computing pki and ci pair | 114.8651 | |
Sending pki and ci to the server | 1117.554 | ||
Server | TPKC: adding a single pki to PKU and adding a single ci to CU | 4.763361 | |
Computing PKU and CU for all N nodes | N·TPKC | ||
Transmitting PKU and CU to each node | 1263.832 | ||
Learning | Node | Computing the Di and Ei pair | 60.08073 |
Transmitting Di and Ei to the server | 590.122 | ||
Server | TD: Adding a single Di to DU | 0.026612 | |
Computing DU for all N nodes | N·TD | ||
TE: Adding a single Ei to EU | 7.099121 | ||
Computing EU for all N nodes | N·TE | ||
Computing w from DU and EU, including decoding EU | 0.15177 |
Features | The Proposed Model | Masking-Based Aggregation by K. Bonawitz et al. [22] | HE-Based Aggregation by J. Park and H. Lim [15] | MKFHE-Based Aggregation by W. Liu et al. [16] | |
---|---|---|---|---|---|
Privacy-preserving strategies for local parameters | Masking and HE (based on CKKS-HE) | Masking | HE (based on a Distributed Homomorphic Cryptosystem) | HE (based on CKKS-HE) | |
Use of additional third parties | X | X | Computation Provider (CP) | X | |
The number of interactions required for aggregation (Assume that the set of nodes for aggregation has been determined and no dropouts occurred) | Number | 2 | 4 | 4 | 4 |
Interactions | N→S: send local updates S→N: send global update | N→S: send masks for other nodes S→N: distribute masks N→S: transmit masked local update S→N: transmit global update | N→S: transmit encrypted local update S→CP: transmit partially decrypted aggregation CP→S: transmit encrypted sum S→N: transmit encrypted global update | N→S: transmit encrypted local update S→N: transmit encrypted aggregation N→S: transmit partial decryption S→N: transmit global update | |
Type of global update retrieved by the server | Global update | Global update | Encrypted global update | Global update | |
Use of different keys for nodes | O | N/A | O | O | |
Update encryption key when dropout occurs | X | N/A | N/A | O | |
Decryption process | X | N/A | O (each node needs to decrypt the encrypted global update) | O (each node needs to partially decrypt the given aggregation) |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2025 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Park, S.; Lee, J.; Harada, K.; Chi, J. Masking and Homomorphic Encryption-Combined Secure Aggregation for Privacy-Preserving Federated Learning. Electronics 2025, 14, 177. https://doi.org/10.3390/electronics14010177
Park S, Lee J, Harada K, Chi J. Masking and Homomorphic Encryption-Combined Secure Aggregation for Privacy-Preserving Federated Learning. Electronics. 2025; 14(1):177. https://doi.org/10.3390/electronics14010177
Chicago/Turabian StylePark, Soyoung, Junyoung Lee, Kaho Harada, and Jeonghee Chi. 2025. "Masking and Homomorphic Encryption-Combined Secure Aggregation for Privacy-Preserving Federated Learning" Electronics 14, no. 1: 177. https://doi.org/10.3390/electronics14010177
APA StylePark, S., Lee, J., Harada, K., & Chi, J. (2025). Masking and Homomorphic Encryption-Combined Secure Aggregation for Privacy-Preserving Federated Learning. Electronics, 14(1), 177. https://doi.org/10.3390/electronics14010177