Graphical Minimax Game and On-Policy Reinforcement Learning for Consensus of Leaderless Multi-Agent Systems *

In this paper, we study the adaptive optimal consensus control of leaderless multi-agent systems (MASs) with heterogeneous dynamics. First, the consensus control problem is converted into a graphical minimax game problem and the corresponding algebraic Riccati equation (ARE) for each agent is obtained. Then, an on-policy reinforcement learning algorithm is proposed to online learn the optimal control policy without requiring the system dynamics. A certain rank condition is established to guarantee the convergence of the proposed online learning algorithm to the unique solution of the ARE. Finally, the effectiveness of the proposed algorithm is demonstrated through a numerical simulation.