TY - GEN
T1 - UAVs Pursuit-Evasion Game Based on M3DDPG with Adversarial Prioritized Experience Replay
AU - Jin, Long
AU - Sun, Zhongqi
AU - Du, Changkun
AU - Xia, Yuanqing
N1 - Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2027.
PY - 2027
Y1 - 2027
N2 - The pursuit and evasion game problem for UAVs (Unmanned Aerial Vehicles) has attracted increasing interest from researchers due to its promising applications in many fields. Traditional methods are effectively difficult to deal with the dynamic interaction of strategies, and adaptability is seriously insufficient. The Minimax Deep Deterministic Policy Gradient (M3DDPG) algorithm offers a minimax optimization framework and an adversarial training process, making it well-suited for gaming tasks. However, the M3DDPG algorithm does not converge fast enough in rapidly dynamic environments. In this paper, we propose an M3DDPG-APER (M3DDPG-Adversarial Prioritized Experience Replay) algorithm, which integrates a gradient-based paradigm of adversarial prioritization into a pursuit-evasion game. The method prioritizes adversarial sample replay by calculating the gradient importance of adversarial samples, which leads to a more efficient use of key experiences and accelerates the learning of adversarial strategies, scenarios, and improves convergence while satisfying the physical constraints of UAVs.
AB - The pursuit and evasion game problem for UAVs (Unmanned Aerial Vehicles) has attracted increasing interest from researchers due to its promising applications in many fields. Traditional methods are effectively difficult to deal with the dynamic interaction of strategies, and adaptability is seriously insufficient. The Minimax Deep Deterministic Policy Gradient (M3DDPG) algorithm offers a minimax optimization framework and an adversarial training process, making it well-suited for gaming tasks. However, the M3DDPG algorithm does not converge fast enough in rapidly dynamic environments. In this paper, we propose an M3DDPG-APER (M3DDPG-Adversarial Prioritized Experience Replay) algorithm, which integrates a gradient-based paradigm of adversarial prioritization into a pursuit-evasion game. The method prioritizes adversarial sample replay by calculating the gradient importance of adversarial samples, which leads to a more efficient use of key experiences and accelerates the learning of adversarial strategies, scenarios, and improves convergence while satisfying the physical constraints of UAVs.
KW - M3DDPG
KW - Minimax optimization
KW - Pursuit-evasion game
UR - https://www.scopus.com/pages/publications/105050753448
U2 - 10.1007/978-981-92-4392-1_15
DO - 10.1007/978-981-92-4392-1_15
M3 - Conference contribution
AN - SCOPUS:105050753448
SN - 9789819243914
T3 - Lecture Notes in Electrical Engineering
SP - 170
EP - 182
BT - Proceedings of 2025 9th Chinese Conference on Swarm Intelligence and Cooperative Control - Swarm Decision and Planning Technologies
A2 - Li, Xiaoduo
A2 - Li, Yichen
A2 - Feng, Zhi
PB - Springer Science and Business Media Deutschland GmbH
T2 - 9th Chinese Conference on Swarm Intelligence and Cooperative Control, CCSICC 2025
Y2 - 31 October 2025 through 3 November 2025
ER -