Scalable Cooperative Decision-Making in Multi-UAV Confrontations: An Attention-Based Multiagent Actor–Critic Approach

With the increasing use of unmanned aerial vehicles (UAVs) in military operations, autonomous cooperative decision-making for multiple UAVs in aerial confrontations has become a critical research challenge. This article presents an attention-based multiagent actor–critic (AMAAC) algorithm for UAVs aerial confrontation decision-making. The algorithm combines multihead attention and self-play within the centralized training-distributed execution framework, extending the actor–critic approach based on the missile hit probability prediction model to multi-UAV scenarios. An observations-of-fighters encoder and a centralized critic network based on the attention mechanism are introduced to adapt to varying number of UAVs (different scales) and enhance training performance. In addition, self-play-based extended training is used to generalize offensive and defensive strategies from small-scale aerial confrontations to larger scenarios. Experimental results demonstrate that the AMAAC algorithm achieves superior training effectiveness, and the strategies it produces perform well across various confrontation scales, even beyond the training scenario’s scale. Compared to other decision-making algorithms, such as multiagent proximal policy optimization, multiagent hierarchical policy gradient, and the state-event-condition-action algorithm, the AMAAC-trained strategies yield higher win ratios and kill-death ratios in different scenarios, validating the algorithm’s effectiveness and scalability.