Distributed Deep Reinforcement Learning for Resource Allocation in Digital Twin Networks

With the rapid growth of the wireless network scale and the aggressive development of communication technology, the communication network connection is required to drift to digits in order to ameliorate the network efficiency. Digital twin (DT) is one of the most promising techniques, which promotes the digital transition of communication networks by establishing mappings between virtual models and physical objects. Nevertheless, due to the limitation and heterogeneity of equipment resources, it is a great challenge to provide efficient network resource allocation. To solve this problem, the authors propose a novel network paradigm based on digital twin to build the topology and model of the communication system. Then a distributed deep reinforcement learning (DRL) method is designed to dispose the problem of resource allocation in cellular networks, and an online–offline learning framework is proposed. Firstly, the offline training is carried out in the simulation environment, and the DRL algorithm is applied to train the deep neural network (DNN). Secondly, in the process of online learning, the real data are further utilized to fine-tune the DNN. Numerical results illustrate the superiority of the proposed method in terms of average system capacity. In the case of different user densities, the performance of the proposed algorithm has more advantages than that of benchmark algorithms and has better generalization ability.