Formal logic enabled personalized federated learning through property inference
Article No.: 1214, Pages 10882 - 10890
Abstract
Recent advancements in federated learning (FL) have greatly facilitated the development of decentralized collaborative applications, particularly in the domain of Artificial Intelligence of Things (AIoT). However, a critical aspect missing from the current research landscape is the ability to enable data-driven client models with symbolic reasoning capabilities. Specifically, the inherent heterogeneity of participating client devices poses a significant challenge, as each client exhibits unique logical reasoning properties. Failing to consider these device-specific specifications can result in missing critical properties in the client predictions, leading to suboptimal performance. This work proposes a new training paradigm that leverages temporal logic reasoning to address this issue. Our approach enhances the training process by incorporating mechanically generated logic expressions for each FL client. Additionally, we develop aggregation clusters and a partitioning algorithm to effectively group clients based on the alignment of their temporal reasoning properties. We evaluate the proposed method on two tasks: a real-world traffic volume prediction task consisting of sensory data from fifteen states and a smart city multi-task prediction utilizing synthetic data. The evaluation results exhibit clear improvements, with performance accuracy improved by up to 54% across all sequential prediction models.
References
[1]
An, Z.; and Ma, M. 2023. Guiding Federated Learning with Inferenced Formal Logic Properties. In Proceedings of the ACM/IEEE 14th International Conference on Cyber-Physical Systems (with CPS-IoT Week 2023), ICCPS '23, 274-275. New York, NY, USA: Association for Computing Machinery. ISBN 9798400700361.
[2]
Arivazhagan, M. G.; Aggarwal, V.; Singh, A. K.; and Choudhary, S. 2019. Federated learning with personalization layers. arXiv preprint arXiv:1912.00818.
[3]
Bartocci, E.; Mateis, C.; Nesterini, E.; and Nickovic, D. 2022. Survey on mining signal temporal logic specifications. Information and Computation, 104957.
[4]
Chen, D.; Gao, D.; Kuang, W.; Li, Y.; and Ding, B. 2022. pFL-Bench: A Comprehensive Benchmark for Personalized Federated Learning. arXiv preprint arXiv:2206.03655.
[5]
Collins, L.; Hassani, H.; Mokhtari, A.; and Shakkottai, S. 2021. Exploiting shared representations for personalized federated learning. In International Conference on Machine Learning, 2089-2099. PMLR.
[6]
Deng, Y.; Kamani, M. M.; and Mahdavi, M. 2020. Adaptive personalized federated learning. arXiv preprint arXiv:2003.13461.
[7]
Diligenti, M.; Roychowdhury, S.; and Gori, M. 2017. Integrating prior knowledge into deep learning. In 2017 16th IEEE international conference on machine learning and applications (ICMLA), 920-923. IEEE.
[8]
Donze, A.; and Maler, O. 2010. Robust satisfaction of temporal logic over real-valued signals. In Formal Modeling and Analysis of Timed Systems: 8th International Conference, FORMATS 2010, Klosterneuburg, Austria, September 8-10, 2010. Proceedings 8, 92-106. Springer.
[9]
Fallah, A.; Mokhtari, A.; and Ozdaglar, A. 2020. Personalized federated learning: A meta-learning approach. arXiv preprint arXiv:2002.07948.
[10]
FHWA. 2016. Highway Performance Monitoring System Field Manual. https://www.fhwa.dot.gov/policyinformation/hpms/fieldmanual/hpms_field_manual_dec2016.pdf. Accessed: 2024-01-24.
[11]
Ghosh, A.; Chung, J.; Yin, D.; and Ramchandran, K. 2020. An efficient framework for clustered federated learning. Advances in Neural Information Processing Systems, 33: 19586-19597.
[12]
Hinton, G.; Vinyals, O.; and Dean, J. 2015. Distilling the Knowledge in a Neural Network. arXiv:1503.02531.
[13]
Hu, Z.; Ma, X.; Liu, Z.; Hovy, E.; and Xing, E. 2020. Harnessing Deep Neural Networks with Logic Rules. arXiv:1603.06318.
[14]
Jha, S.; Tiwari, A.; Seshia, S. A.; Sahai, T.; and Shankar, N. 2017. Telex: Passive stl learning using only positive examples. In International Conference on Runtime Verification, 208-224. Springer.
[15]
Jia, X.; Willard, J.; Karpatne, A.; Read, J. S.; Zwart, J. A.; Steinbach, M.; and Kumar, V. 2021. Physics-guided machine learning for scientific discovery: An application in simulating lake temperature profiles. ACM/IMS Transactions on Data Science, 2(3): 1-26.
[16]
Krajzewicz, D.; Hertkorn, G.; Rossel, C.; and Wagner, P. 2002. SUMO (Simulation of Urban MObility)-an open-source traffic simulation. In Proceedings of the 4th middle East Symposium on Simulation and Modelling (MESM20002), 183-187.
[17]
Li, T.; Hu, S.; Beirami, A.; and Smith, V. 2021a. Ditto: Fair and robust federated learning through personalization. In International Conference on Machine Learning, 6357-6368. PMLR.
[18]
Li, T.; Sahu, A. K.; Talwalkar, A.; and Smith, V. 2020. Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3): 50-60.
[19]
Li, X.; Jiang, M.; Zhang, X.; Kamp, M.; and Dou, Q. 2021b. Fedbn: Federated learning on non-iid features via local batch normalization. arXiv preprint arXiv:2102.07623.
[20]
Ma, M.; Bartocci, E.; Lifland, E.; Stankovic, J. A.; and Feng, L. 2021a. A novel spatial-temporal specification-based monitoring system for smart cities. IEEE Internet of Things Journal, 8(15): 11793-11806.
[21]
Ma, M.; Gao, J.; Feng, L.; and Stankovic, J. 2020. STLnet: Signal temporal logic enforced multivariate recurrent neural networks. Advances in Neural Information Processing Systems, 33: 14604-14614.
[22]
Ma, M.; Preum, S. M.; Ahmed, M. Y.; Tarneberg, W.; Hendawi, A.; and Stankovic, J. A. 2019. Data sets, modeling, and decision making in smart cities: A survey. ACM Transactions on Cyber-Physical Systems, 4(2): 1-28.
[23]
Ma, M.; Preum, S. M.; and Stankovic, J. A. 2017. Cityguard: A watchdog for safety-aware conflict detection in smart cities. In Proceedings of the Second International Conference on Internet-of-Things Design and Implementation, 259-270.
[24]
Ma, M.; Stankovic, J.; Bartocci, E.; and Feng, L. 2021b. Predictive monitoring with logic-calibrated uncertainty for cyber-physical systems. ACM Transactions on Embedded Computing Systems (TECS), 20(5s): 1-25.
[25]
Ma, M.; Stankovic, J. A.; and Feng, L. 2018. Cityresolver: a decision support system for conflict resolution in smart cities. In 2018 ACM/IEEE 9th International Conference on Cyber-Physical Systems (ICCPS), 55-64. IEEE.
[26]
Ma, M.; Stankovic, J. A.; and Feng, L. 2021. Toward formal methods for smart cities. Computer, 54(9): 39-48.
[27]
Maler, O.; and Nickovic, D. 2004. Monitoring temporal properties of continuous signals. In Formal Techniques, Modelling and Analysis of Timed and Fault-Tolerant Systems, 152-166. Springer.
[28]
Mansour, Y.; Mohri, M.; Ro, J.; and Suresh, A. T. 2020. Three approaches for personalization with applications to federated learning. arXiv preprint arXiv:2002.10619.
[29]
McMahan, B.; Moore, E.; Ramage, D.; Hampson, S.; and y Arcas, B. A. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, 1273-1282. PMLR.
[30]
Mohri, M.; Sivek, G.; and Suresh, A. T. 2019. Agnostic federated learning. In International Conference on Machine Learning, 4615-4625. PMLR.
[31]
Muralidhar, N.; Islam, M. R.; Marwah, M.; Karpatne, A.; and Ramakrishnan, N. 2018. Incorporating prior domain knowledge into deep neural networks. In 2018 IEEE international conference on big data (big data), 36-45. IEEE.
[32]
Nguyen, D. C.; Ding, M.; Pathirana, P. N.; Seneviratne, A.; Li, J.; and Poor, H. V. 2021. Federated learning for internet of things: A comprehensive survey. IEEE Communications Surveys & Tutorials, 23(3): 1622-1658.
[33]
Preum, S. M.; Munir, S.; Ma, M.; Yasar, M. S.; Stone, D. J.; Williams, R.; Alemzadeh, H.; and Stankovic, J. A. 2021. A review of cognitive assistants for healthcare: Trends, prospects, and future directions. ACM Computing Surveys (CSUR), 53(6): 1-37.
[34]
Tan, A. Z.; Yu, H.; Cui, L.; and Yang, Q. 2022. Towards personalized federated learning. IEEE Transactions on Neural Networks and Learning Systems.
[35]
Von Rueden, L.; Mayer, S.; Beckh, K.; Georgiev, B.; Giesselbach, S.; Heese, R.; Kirsch, B.; Pfrommer, J.; Pick, A.; Ramamurthy, R.; et al. 2021. Informed Machine Learning-A taxonomy and survey of integrating prior knowledge into learning systems. IEEE Transactions on Knowledge and Data Engineering, 35(1): 614-633.
[36]
Yu, T.; Bagdasaryan, E.; and Shmatikov, V. 2020. Salvaging federated learning by local adaptation. arXiv preprint arXiv:2002.04758.
Index Terms
- Formal logic enabled personalized federated learning through property inference
Index terms have been assigned to the content through auto-classification.
Recommendations
Poisoning-Assisted Property Inference Attack Against Federated Learning
Federated learning (FL) has emerged as an ideal privacy-preserving learning technique which can train a global model in a collaborative way while preserving the private data in the local. However, recent advances have demonstrated that FL is still ...
Invariance in property testing
Property testingProperty testing considers the task of testing rapidly (in particular, with very few samples into the data), if some massive data satisfies some given property, or is far from satisfying the property. For "global properties", i.e., properties that ...
Invariance in property testing
Property testingProperty testing considers the task of testing rapidly (in particular, with very few samples into the data), if some massive data satisfies some given property, or is far from satisfying the property. For "global properties", i.e., properties that ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
February 2024
23861 pages
ISBN:978-1-57735-887-9
Copyright © 2024 Association for the Advancement of Artificial Intelligence.
Sponsors
- Association for the Advancement of Artificial Intelligence
Publisher
AAAI Press
Publication History
Published: 20 February 2024
Qualifiers
- Research-article
- Research
- Refereed limited
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025