Online Reinforcement Learning for Model-Free Secure Formation Control Under DoS Attacks

This paper develops an online reinforcement learning framework to address secure formation control challenges for unknown air-ground systems under denial-of-service (DoS) attacks. We first propose a resilient distributed observer against multi-channel DoS attacks, which incorporates channel-dependent decay rates specifically designed for multi-channel DoS mitigation. This resilient observer ensures secure state estimation when decay rates meet specified criteria, and simultaneously provides output tracking references for followers in integrated air-ground formation control systems. Building on this observer design, we further develop a distributed feedforward-feedback formation control policy via an initial excitation-based online reinforcement learning algorithm, enabling data-driven air-ground formation control without prior system knowledge. Compared to conventional learning algorithms addressing similar problems, our proposed method effectively overcomes several critical limitations, including the persistent excitation requirement, memory-intensive delayed-window integral computations, full-rank matrix conditions, and historical data storage dependencies. Finally, numerical simulations are presented to validate the effectiveness of the theoretical results.