Joint Path Planning and Transmission Scheduling for Multi-UAV Data Collection

Due to their high adaptability and maneuverability, unmanned aerial vehicles (UAVs) can be used to collect data from ground sensor nodes (SNs) in the Internet of Things (IoTs). However, the number of SNs is increasing and the distribution becomes more complex, thus making it difficult for path planning and transmission scheduling of UAVs. In this paper, we consider a problem of maximizing the data volume collected from a large number of ground SNs with multiple UAVs. Limited by the observation capability, each UAV can only observe information from a proportion of the SNs during the collecting process, which makes it difficult to find an optimal solution for the global maximization problem. To solve this problem, we first formulate the optimization problem as a decentralized partially observable Markov decision process problem. Then, a novel multi-agent Deep Reinforcement Learning algorithm with centralized training and decentralized execution structure is proposed for the solution. Specifically, each UAV is equipped with a two-layer Hierarchical Deep Q Network (HDQN) to independently select actions for data collection based on its local observation. The first DQN layer of HDQN selects actions for transmission scheduling, and the second DQN layer selects actions for path planning. Then, a mixing network is utilized to estimate the joint action-value using global information and train the UAVs. Extensive experiments demonstrate that the proposed HQMIX algorithm achieves a maximum performance enhancement of 18.6% in the specified simulation environment in comparison with the baseline algorithms.