Robust MLLM-Based Decision Making for Swarm Confrontation Under Uncertain Constraints

Global decision-making is essential in swarm confrontation, enabling independent agents to establish strategic advantages through cooperation. However, execution of strategy in realistic scenarios faces significant challenges due to pervasive uncertainties. Traditional rule-based and reinforcement learning methods that rely on precise state information often lack the generalization capability to handle these uncertainties. Here, we propose a global decision-making method for swarm confrontation based on Multi-Modal Large Language Models (MLLMs) to address environmental uncertainties. This approach leverages the advanced understanding and reasoning of MLLMs to perform strategic planning, inferring from multi-modal observational data to make robust decisions. Furthermore, to mitigate the inference latency of MLLMs during implementation, we introduce a trajectory prediction mechanism for compensation. Extensive experiments demonstrate that our method outperforms the baseline methods in terms of win rate and strategic efficiency. In addition, our method exhibits strong adaptability and robustness under various uncertainty constraints.