TY - JOUR
T1 - Efficient Hierarchical Federated Learning With Pareto-Optimal Bi-Level Reinforcement Learning
AU - Manjang, Ousman
AU - Zhai, Yanlong
AU - Shen, Jun
AU - Sarwar, Adil
AU - He, Xutian
AU - Wang, Huan
AU - Zhu, Liehuang
N1 - Publisher Copyright:
© 2014 IEEE.
PY - 2025
Y1 - 2025
N2 - Hierarchical Federated Learning (HFL) has emerged as a popular federated learning method by introducing additional aggregation levels using intermediate edge servers. The performance of HFL approaches is contingent upon the aggregation frequency and the number of intermediate aggregation rounds. Existing approaches mainly focus on optimizing the aggregation frequency only, neglecting the impact of intermediate aggregation rounds on training performance. On the other hand, these methods also fail to consider the multidimensional effects of aggregation frequency, consequently focusing only on the performance accuracy while overlooking the detrimental effects on both the communication costs and training latency. This paper introduces HFL-PBRL, a novel HFL framework that employs a bi-level reinforcement learning (RL) based algorithm to jointly optimize aggregation frequencies and rounds across edge servers. This algorithm is accompanied by a Pareto-efficient multi-objective optimization approach to strike an optimal trade-off among model accuracy, communication cost and convergence time. The varied aggregation frequencies and rounds might introduce inconsistencies; therefore, we employ a hierarchical pullback mechanism that iteratively pulls the client models toward a synchronized anchor model, ensuring effective divergence control. Furthermore, we devise a harmonic weight assignment strategy that dynamically adjusts the aggregation weights of each model based on their current, historical, and anticipated divergence, addressing the model fluctuations and asynchrony. Extensive evaluations demonstrate that HFL-PBRL consistently achieves high model accuracy and faster convergence with minimal communication costs compared to baselines and SOTA.
AB - Hierarchical Federated Learning (HFL) has emerged as a popular federated learning method by introducing additional aggregation levels using intermediate edge servers. The performance of HFL approaches is contingent upon the aggregation frequency and the number of intermediate aggregation rounds. Existing approaches mainly focus on optimizing the aggregation frequency only, neglecting the impact of intermediate aggregation rounds on training performance. On the other hand, these methods also fail to consider the multidimensional effects of aggregation frequency, consequently focusing only on the performance accuracy while overlooking the detrimental effects on both the communication costs and training latency. This paper introduces HFL-PBRL, a novel HFL framework that employs a bi-level reinforcement learning (RL) based algorithm to jointly optimize aggregation frequencies and rounds across edge servers. This algorithm is accompanied by a Pareto-efficient multi-objective optimization approach to strike an optimal trade-off among model accuracy, communication cost and convergence time. The varied aggregation frequencies and rounds might introduce inconsistencies; therefore, we employ a hierarchical pullback mechanism that iteratively pulls the client models toward a synchronized anchor model, ensuring effective divergence control. Furthermore, we devise a harmonic weight assignment strategy that dynamically adjusts the aggregation weights of each model based on their current, historical, and anticipated divergence, addressing the model fluctuations and asynchrony. Extensive evaluations demonstrate that HFL-PBRL consistently achieves high model accuracy and faster convergence with minimal communication costs compared to baselines and SOTA.
KW - aggregation frequency
KW - aggregation rounds
KW - aggregation weight
KW - Hierarchical federated learning
UR - http://www.scopus.com/inward/record.url?scp=105005998271&partnerID=8YFLogxK
U2 - 10.1109/JIOT.2025.3572900
DO - 10.1109/JIOT.2025.3572900
M3 - Article
AN - SCOPUS:105005998271
SN - 2327-4662
JO - IEEE Internet of Things Journal
JF - IEEE Internet of Things Journal
ER -