A dynamic control decision approach for fixed-wing aircraft games via hybrid action reinforcement learning

Autonomous decision-making is crucial for aircraft to achieve quick victories in diverse scenarios. Based on a 6-degree-of-freedom aircraft model, this paper proposes a decoupled guidance and control theory for autonomous aircraft maneuvering, distinguishing between close and long-range engagements. We introduce a method for heading attitude control to enhance stability during close-range interactions and a speed-based adaptive grid model for precise waypoint control in mid-to-long-range engagements. The paper transforms dynamic aircraft interactions into a Markov decision process and presents a hybrid discrete and continuous action reinforcement learning approach. This unified learning framework offers enhanced generalization and learning speed for dynamic aircraft adversarial processes. Experimental results indicate that in a symmetric environment, our approach rapidly achieves Nash equilibrium, securing over a 10% advantage. In unmanned aerial aircraft game control with higher maneuverability, the probability of gaining a situational advantage increases by more than 40%. Compared to similar methods, our approach demonstrates superior effectiveness in decision optimization and adversarial success probability. Furthermore, we validate the algorithm’s robustness and adaptability in an asymmetric environment, showcasing its promising application potential in collaborative control of aircraft clusters.