An improved PPO-GTrXL algorithm for missile evasion in partially observable air combat

Missile evasion is a critical aspect of aircraft survivability in modern air combat, particularly under the threat of agile, high-speed missiles with advanced guidance systems. However, conventional rule-based or optimal control approaches often fail to generalize in dynamic and partially observable environments. To address this challenge, we propose a novel missile evasion framework that integrates Proximal Policy Optimization (PPO) with Gated Transformer-XL (GTrXL). A high-fidelity simulation environment is developed, incorporating a realistic aircraft dynamics model and a proportional navigation-guided missile. We design a customized reward function to encourage survival-oriented behavior while adhering to flight constraints. The proposed PPO-GTrXL agent demonstrates enhanced adaptability and decision-making under uncertainty. Experimental results show that PPO-GTrXL improves the average return from -560.18 to 407.05 and raises the win rate from 63.3% to 85.7%, while also reducing performance variance compared to baseline PPO. These findings highlight the robustness and effectiveness of our method in complex aerial engagement scenarios.