EntroCFL: Entropy-Based Clustered Federated Learning With Incentive Mechanism

Federated learning (FL) emerged as a machine learning approach in situations where the privacy of sensitive data needs to be protected. Within the FL framework, clients collaborate to train a shared global model using their individual data by sending the model parameters to a central server, all while keeping their private data localized. Albeit its advantage, FL still faces certain limitations, such as clients may lack the motivation to participate in training, and the heterogeneous data distribution among clients can slow down the model convergence rate and degrade the model accuracy. In the light of the above mentioned considerations, we introduce entropy-based clustered FL (EntroCFL) with incentive mechanism, a two-layer clustered FL (CFL) model to jointly address the incentive mechanism and model training performance issues with heterogeneous clients. In Layer I, the server designs the payments to the clients to minimize its cost, including the training accuracy loss and the payment to clients, based on which the clients determine their training datasizes to maximize their own utilities. In Layer II, we introduce an entropy-based clustering method that is implemented based on the clients' strategies in Layer I. Unlike conventional CFL methods that rely solely on the cosine similarity between clients' parameter gradients, EntroCFL introduces a novel clustering discriminant which takes both angle and magnitude of clients' parameter gradients into consideration. Simulation experiments are conducted to compare EntroCFL with conventional methods, such as FedAvg on the MNIST, EMNIST, and FMNIST datasets. The results validate the superiority of EntroCFL in terms of experimental accuracy, robustness, and economic efficiency.