TD3-Based Trajectory Design for UAV-Assisted Communication in Unknown Environment

With the widespread use of unmanned aerial vehicles (UAVs), leveraging UAVs to provide communication services in unknown environments when terrestrial cellular networks are unavailable has become an increasingly critical issue. In this paper, we investigate the trajectory design for a UAV offers communication services to ground users (GUs) in an unknown environment, while simultaneously using the echoes to characterize the environmental topology for obstacle avoidance. By modeling the trajectory optimization problem as a Markov Decision Process (MDP), we employ deep reinforcement learning (DRL) to simulate the UAV's flight in an unknown environment and identify the optimal strategy through repeated trials. In particular, we propose a twin-delayed deep deterministic policy gradient (TD3)-based trajectory design (TD3BTD) method to minimize the time required to provide communication services to all GUs. Simulation results demonstrate that the proposed TD3BTD significantly reduces the mission execution time.