Hybrid Hierarchical Reinforcement Learning for Nonlinear Morphing Aircraft Control

Morphing aircraft can adjust geometric configuration during flight, which makes tracking control more difficult than in fixed configuration aircraft. To address this problem, a hierarchical framework integrating twin delayed deep deterministic policy gradient (TD3) algorithm and incremental nonlinear dynamic inversion (INDI) technique is proposed. The TD3 algorithm generates high-level decisions for the coupled loops, while the INDI technique performs fast actuator tracking at the lower layer. The design separates decision making from command execution and improves the coordination of the coupled control process during morphing. Meanwhile, a unified reward formulation is designed to guide policy learning toward accurate and stable control behavior. Simulation results show that the proposed framework achieves favorable performance in tracking accuracy, response smoothness, and coordination of the coupled system.