Boosting Semi-Supervised Federated Learning with Model Personalization and Client-Variance-Reduction

Shuai Wang*, Yanqing Xu, Yanli Yuan, Xiuhua Wang, Tony Q.S. Quek

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review

4 Citations (Scopus)

Abstract

Recently, federated learning (FL) has been increasingly appealing in distributed signal processing and machine learning. Nevertheless, the practical challenges of label deficiency and client heterogeneity form a bottleneck to its wide adoption. Although numerous efforts have been devoted to semi- supervised FL, most of the adopted algorithms follow the same spirit as FedAvg, thus heavily suffering from the adverse effects caused by client heterogeneity. In this paper, we boost the semi-supervised FL by addressing the issue using model personalization and client-variance-reduction. In particular, we propose a novel and unified problem formulation based on pseudo-labeling and model interpolation. We then propose an effective algorithm, named FedCPSL, which judiciously adopts the schemes of a novel momentum-based client- variance-reduction and normalized averaging. Convergence property of FedCPSL is analyzed and shows that FedCPSL is resilient to client heterogeneity and obtains a sublinear convergence rate. Experimental results on image classification tasks are also presented to demonstrate the efficacy of FedCPSL over the benchmark algorithms.

Keywords

  • Semi-supervised learning
  • client variance reduction
  • federated learning
  • model personalization

Fingerprint

Dive into the research topics of 'Boosting Semi-Supervised Federated Learning with Model Personalization and Client-Variance-Reduction'. Together they form a unique fingerprint.

Cite this

Wang, S., Xu, Y., Yuan, Y., Wang, X., & Quek, T. Q. S. (2023). Boosting Semi-Supervised Federated Learning with Model Personalization and Client-Variance-Reduction. Proceedings - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing. https://doi.org/10.1109/ICASSP49357.2023.10096678