Adaptive Data Collaboration Based on Multi-Agent Reinforcement Learning in Internet of Things

Decentralized Federated Learning (DFL) has become a prominent privacy-preserving data collaboration paradigm in Internet of Things (IoT), crucial for advancing Artificial Intelligence applications. However, the intricate dynamics of wireless environments and the heterogeneity among collaborative IoT nodes present great challenges to the learning efficiency of conventional DFL processes. Therefore, the development of an adaptive collaboration strategy of heterogeneous nodes is of prominent importance to facilitate efficient DFL in IoT networks. In this paper, we introduce an adaptive data collaboration mechanism based on multi-agent reinforcement learning (MADC) that enables heterogeneous nodes to learn tailored collaboration strategies in dynamic IoT networks. In MADC design, we tackle inherent challenges such as vast action space and overestimation by proposing the mean filed representation and dual critic network-based approximation methods. Extensive numerical results demonstrate that the proposed MADC outperforms in terms of model accuracy, learning efficiency, and communication cost compared to baselines.