A systematic survey on federated semi-supervised learning
AUTHORs: 
Zixing Song, Xiangli Yang, Yifei Zhang, Xinyu Fu, Zenglin Xu, Irwin KingAuthors Info & Claims
IJCAI '24: Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence
Article No.: 911, Pages 8244 - 8252
Abstract
Federated learning (FL) revolutionizes distributed machine learning by enabling devices to collaboratively learn a model while maintaining data privacy. However, FL usually faces a critical challenge with limited labeled data, making semi-supervised learning (SSL) crucial for utilizing abundant unlabeled data. The integration of SSL within the federated framework gives rise to federated semi-supervised learning (FSSL), a novel approach that exploits unlabeled data across devices without compromising privacy. This paper systematically explores FSSL, shedding light on its four basic problem settings that commonly appear in real-world scenarios. By examining the unique challenges, generic solutions, and representative methods tailored for each setting of FSSL, we aim to provide a cohesive overview of the current state of the art and pave the way for future research directions in this promising field.
References
[1]
Liwei Che, Zewei Long, Jiaqi Wang, Yaqing Wang, Houping Xiao, and Fenglong Ma. Fedtrinet: A pseudo labeling method with three players for federated semi-supervised learning. In IEEE BigData, 2021.
[2]
Enmao Diao, Jie Ding, and Vahid Tarokh. Semifl: Semi-supervised federated learning for unlabeled clients with alternate training. In NeurIPS, 2022.
[3]
Chenyou Fan, Junjie Hu, and Jianwei Huang. Private semi-supervised federated learning. In IJCAI, pages 2009-2015. ijcai.org, 2022.
[4]
Chong Fu, Xuhong Zhang, Shouling Ji, Jinyin Chen, Jingzheng Wu, Shanqing Guo, Jun Zhou, Alex X. Liu, and Ting Wang. Label inference attacks against vertical federated learning. In USENIX Security Symposium, pages 1397-1414, 2022.
[5]
Sohei Itahara, Takayuki Nishio, Yusuke Koda, Masahiro Morikura, and Koji Yamamoto. Distillation-based semi-supervised federated learning for communication-efficient collaborative training with non-iid private data. IEEE TMC, 2023.
[6]
Wonyong Jeong, Jaehong Yoon, Eunho Yang, and Sung Ju Hwang. Federated semi-supervised learning with inter-client consistency & disjoint learning. In ICLR, 2021.
[7]
Yilun Jin, Yang Liu, Kai Chen, and Qiang Tang. Federated learning without full labels: A survey. IEEE Data Eng. Bull., 46(1):27-51, 2023.
[8]
Yan Kang, Yang Liu, and Xinle Liang. Fedcvt: Semi-supervised vertical federated learning with cross-view training. ACM Trans. Intell. Syst. Technol., 13(4):64:1-64:16, 2022.
[9]
Sangmook Kim, Wonyoung Shin, Soohyuk Jang, Hwanjun Song, and Se-Young Yun. Fedrn: Exploiting k-reliable neighbors towards robust federated learning. In CIKM, pages 972-981. ACM, 2022.
[10]
Dong-Hyun Lee et al. Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks. In Workshop on challenges in representation learning, ICML, 2013.
[11]
Chongxuan Li, Taufik Xu, Jun Zhu, and Bo Zhang. Triple generative adversarial nets. In NIPS, pages 4088-4098, 2017.
[12]
Ming Li, Qingli Li, and Yan Wang. Class balanced adaptive pseudo labeling for federated semi-supervised learning. In CVPR, pages 16292-16301, 2023.
[13]
Qinbin Li, Zeyi Wen, Zhaomin Wu, Sixu Hu, Naibo Wang, Yuan Li, Xu Liu, and Bingsheng He. A survey on federated learning systems: Vision, hype and reality for data privacy and protection. IEEE Trans. Knowl. Data Eng., 35(4):3347-3366, 2023.
[14]
Xiaoxiao Liang, Yiqun Lin, Huazhu Fu, Lei Zhu, and Xiaomeng Li. Rscfed: Random sampling consensus federated semi-supervised learning. In CVPR, pages 10144-10153. IEEE, 2022.
[15]
Haowen Lin, Jian Lou, Li Xiong, and Cyrus Shahabi. Semifed: Semi-supervised federated learning with consistency and pseudo-labeling. CoRR, abs/2108.09412, 2021.
[16]
Xinyang Lin, Hanting Chen, Yixing Xu, Chao Xu, Xiaolin Gui, Yiping Deng, and Yunhe Wang. Federated learning with positive and unlabeled data. In ICML. PMLR, 2022.
[17]
Quande Liu, Hongzheng Yang, Qi Dou, and Pheng-Ann Heng. Federated semi-supervised medical image classification via inter-client relation matching. In MICCAI, 2021.
[18]
Xinyi Liu, Linghui Zhu, Shu-Tao Xia, Yong Jiang, and Xue Yang. GDST: global distillation self-training for semi-supervised federated learning. In GLOBECOM, pages 1-6. IEEE, 2021.
[19]
Yi Liu, Xingliang Yuan, Ruihui Zhao, Cong Wang, Dusit Niyato, and Yefeng Zheng. Poisoning semi-supervised federated learning via unlabeled data: Attacks and defenses. 2022.
[20]
Yueen Ma, Zixing Song, Xuming Hu, Jingjing Li, Yifei Zhang, and Irwin King. Graph component contrastive learning for concept relatedness estimation. In AAAI, pages 13362-13370. AAAI Press, 2023.
[21]
Shubham Malaviya, Manish Shukla, Pratik Korat, and Sachin Lodha. Fedfame: A data augmentation free framework based on model contrastive learning for federated semi-supervised learning. In SAC, pages 1114-1121. ACM, 2023.
[22]
Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. Communication-efficient learning of deep networks from decentralized data. In AISTATS, volume 54, 2017.
[23]
Kishore Nandury, Anand Mohan, and Frederick Weber. Cross-silo federated training in the cloud with diversity scaling and semi-supervised learning. In ICASSP, pages 3085-3089. IEEE, 2021.
[24]
Dinh C. Nguyen, Ming Ding, Pubudu N. Pathirana, Aruna Seneviratne, Jun Li, and H. Vincent Poor. Federated learning for internet of things: A comprehensive survey. IEEE Commun. Surv. Tutorials, 23(3):1622-1658, 2021.
[25]
Dinh C. Nguyen, Quoc-Viet Pham, Pubudu N. Pathirana, Ming Ding, Aruna Seneviratne, Zihuai Lin, Octavia A. Dobre, and Won-Joo Hwang. Federated learning for smart healthcare: A survey. ACM Comput. Surv., 55(3):60:1-60:37, 2023.
[26]
Nicolas Papernot, Martín Abadi, Úlfar Erlingsson, Ian J. Goodfellow, and Kunal Talwar. Semi-supervised knowledge transfer for deep learning from private training data. In ICLR, 2017.
[27]
Yanhang Shi, Siguang Chen, and Haijun Zhang. Uncertainty minimization for personalized federated semi-supervised learning. IEEE Trans. Netw. Sci. Eng., 10(2):1060-1073, 2023.
[28]
Zixing Song, Yifei Zhang, and Irwin King. No change, no gain: Empowering graph neural networks with expected model change maximization for active learning. In NeurIPS, 2023.
[29]
Zixing Song, Yifei Zhang, and Irwin King. Optimal block-wise asymmetric graph construction for graph-based semi-supervised learning. In NeurIPS, 2023.
[30]
Zixing Song, Yuji Zhang, and Irwin King. Towards fair financial services for all: A temporal GNN approach for individual fairness on transaction networks. In CIKM, pages 2331-2341. ACM, 2023.
[31]
Zixing Song, Ziqiao Meng, and Irwin King. A diffusion-based pre-training framework for crystal property prediction. In AAAI, pages 8993-9001. AAAI Press, 2024.
[32]
Jingwei Sun, Ziyue Xu, Dong Yang, Vishwesh Nath, Wenqi Li, Can Zhao, Daguang Xu, Yiran Chen, and Holger R. Roth. Communication-efficient vertical federated learning with limited overlapping samples. In ICCV, pages 5180-5189. IEEE, 2023.
[33]
Ye Tao, Ying Li, and Zhonghai Wu. Semigraphfl: Semi-supervised graph federated learning for graph classification. In PPSN (1), volume 13398 of Lecture Notes in Computer Science, pages 474-487, 2022.
[34]
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In NIPS, pages 1195-1204, 2017.
[35]
Lun Wang, Yang Xu, Hongli Xu, Jianchun Liu, Zhiyuan Wang, and Liusheng Huang. Enhancing federated learning with in-cloud unlabeled data. In ICDE, pages 136-149. IEEE, 2022.
[36]
Jiaqi Wang, Shenglai Zeng, Zewei Long, Yaqing Wang, Houping Xiao, and Fenglong Ma. Knowledge-enhanced semi-supervised federated learning for aggregating heterogeneous lightweight clients in iot. In SDM, pages 496-504. SIAM, 2023.
[37]
Shuai Wang, Yanqing Xu, Yanli Yuan, and Tony Q. S. Quek. Towards fast personalized semi-supervised federated learning in edge networks: Algorithm design and theoretical guarantee. IEEE Transactions on Wireless Communications, pages 1-1, 2023.
[38]
Shuai Wang, Yanqing Xu, Yanli Yuan, Xiuhua Wang, and Tony Q. S. Quek. Boosting semi-supervised federated learning with model personalization and client-variance-reduction. In ICASSP, 2023.
[39]
Xiao-Xiang Wei and Hua Huang. Balanced federated semisupervised learning with fairness-aware pseudo-labeling. IEEE TNNLS, 2022.
[40]
Tingjie Wen, Shengjie Zhao, and Rongqing Zhang. Federated semi-supervised learning through a combination of self and cross model ensembling. In IJCNN, pages 1-8. IEEE, 2022.
[41]
Qizhe Xie, Zihang Dai, Eduard H. Hovy, Thang Luong, and Quoc Le. Unsupervised data augmentation for consistency training. In NeurIPS, 2020.
[42]
Yang Xu, Lun Wang, Hongli Xu, Jianchun Liu, Zhiyuan Wang, and Liusheng Huang. Enhancing federated learning with server-side unlabeled data by adaptive client and data selection. IEEE TMC, pages 1-18, 2023.
[43]
Dong Yang, Ziyue Xu, Wenqi Li, Andriy Myronenko, Holger R. Roth, Stephanie A. Harmon, Sheng Xu, Baris Turkbey, Evrim Turkbey, Xiaosong Wang, Wentao Zhu, Gianpaolo Carrafiello, Francesca Patella, Maurizio Cariati, Hirofumi Obinata, Hitoshi Mori, Kaku Tamura, Peng An, Bradford J. Wood, and Daguang Xu. Federated semi-supervised learning for COVID region segmentation in chest CT using multi-national data from china, italy, japan. Medical Image Anal., 2021.
[44]
Nan Yang, Dong Yuan, Yuning Zhang, Yongkun Deng, and Wei Bao. Asynchronous semi-supervised federated learning with provable convergence in edge computing. IEEE Netw., 36(5):136-143, 2022.
[45]
Xiangli Yang, Zixing Song, Irwin King, and Zenglin Xu. A survey on deep semi-supervised learning. IEEE TKDE, 35(9):8934-8954, 2023.
[46]
Yuhang Yao, Weizhao Jin, Srivatsan Ravi, and Carlee Joe-Wong. FedGCN: Convergence-communication tradeoffs in federated training of graph convolutional networks. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[47]
Xuefei Yin, Yanming Zhu, and Jiankun Hu. A comprehensive survey of privacy-preserving federated learning: A taxonomy, review, and future directions. ACM Comput. Surv., 54(6):131:1-131:36, 2022.
[48]
Zhe Zhang, Shiyao Ma, Jiangtian Nie, Yi Wu, Qiang Yan, Xiaoke Xu, and Dusit Niyato. Semi-supervised federated learning with non-iid data: Algorithm and system design. In HPCC/DSS/SmartCity/DependSys. IEEE, 2021.
[49]
Zhengming Zhang, Yaoqing Yang, Zhewei Yao, Yujun Yan, Joseph E. Gonzalez, Kannan Ramchandran, and Michael W. Mahoney. Improving semi-supervised federated learning by reducing the gradient diversity of models. In IEEE BigData, 2021.
[50]
Chao Zhang, Fangzhao Wu, Jingwei Yi, Derong Xu, Yang Yu, Jindong Wang, Yidong Wang, Tong Xu, Xing Xie, and Enhong Chen. Non-iid always bad? semi-supervised heterogeneous federated learning with local knowledge enhancement. In CIKM, 2023.
[51]
Jie Zhang, Xiaosong Ma, Song Guo, and Wenchao Xu. Towards unbiased training in federated open-world semi-supervised learning. In ICML, 2023.
[52]
Yifei Zhang, Yankai Chen, Zixing Song, and Irwin King. Contrastive cross-scale graph knowledge synergy. In KDD, pages 3422-3433. ACM, 2023.
[53]
Yifei Zhang, Dun Zeng, Jinglong Luo, Zenglin Xu, and Irwin King. A survey of trustworthy federated learning with perspectives on security, robustness and privacy. In WWW (Companion Volume), 2023.
[54]
Chen Zhao, Zhipeng Gao, Qian Wang, Zijia Mo, and Xinlei Yu. Fedgan: A federated semi-supervised learning from non-iid data. In WASA (2), volume 13472 of Lecture Notes in Computer Science, pages 181-192. Springer, 2022.
Index Terms
- A systematic survey on federated semi-supervised learning
Index terms have been assigned to the content through auto-classification.
Recommendations
A Survey of Semi-Supervised Learning Methods
CIS '08: Proceedings of the 2008 International Conference on Computational Intelligence and Security - Volume 02In traditional machine learning approaches to classification, one uses only a labelled set to train the classifier. Labelled instances however are often difficult, expensive, or time consuming to obtain, as they require the efforts of experienced human ...
Inductive Semi-supervised Multi-Label Learning with Co-Training
KDD '17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data MiningIn multi-label learning, each training example is associated with multiple class labels and the task is to learn a mapping from the feature space to the power set of label space. It is generally demanding and time-consuming to obtain labels for training ...
Semi-supervised federated learning on evolving data streams
AbstractFederated learning allows multiple clients to jointly train a model on their private data without revealing their local data to a centralized server. Thereby, federated learning has attracted increasing attention in recent years, and ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2024 International Joint Conferences on Artificial Intelligence.
Sponsors
- International Joint Conferences on Artifical Intelligence (IJCAI)
Publisher
Unknown publishers
Publication History
Published: 03 August 2024
Qualifiers
- Research-article
- Research
- Refereed limited
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025