UAVs Pursuit-Evasion Game Based on M3DDPG with Adversarial Prioritized Experience Replay

The pursuit and evasion game problem for UAVs (Unmanned Aerial Vehicles) has attracted increasing interest from researchers due to its promising applications in many fields. Traditional methods are effectively difficult to deal with the dynamic interaction of strategies, and adaptability is seriously insufficient. The Minimax Deep Deterministic Policy Gradient (M3DDPG) algorithm offers a minimax optimization framework and an adversarial training process, making it well-suited for gaming tasks. However, the M3DDPG algorithm does not converge fast enough in rapidly dynamic environments. In this paper, we propose an M3DDPG-APER (M3DDPG-Adversarial Prioritized Experience Replay) algorithm, which integrates a gradient-based paradigm of adversarial prioritization into a pursuit-evasion game. The method prioritizes adversarial sample replay by calculating the gradient importance of adversarial samples, which leads to a more efficient use of key experiences and accelerates the learning of adversarial strategies, scenarios, and improves convergence while satisfying the physical constraints of UAVs.