BGC: Multi-agent Group Belief with Graph Clustering

Recent advances have witnessed that value decomposed-based multi-agent reinforcement learning methods make an efficient performance in coordination tasks. Most current methods assume that agents can communicate to assist decisions, which is impractical in some real situations. In this paper, we propose an observation-to-cognition method to enable agents to realize high efficient coordination without communication. Inspired by the neighborhood cognitive consistency (NCC), we introduce the group concept to help agents learn a belief, a type of consensus, to realize that adjacent agents tend to accomplish similar sub-tasks to achieve cooperation. We propose a novel agent structure named Belief in Graph Clustering (BGC) via Graph Attention Network (GAT) to generate agent group belief. In this module, we further utilize an MLP-based module to characterize special agent features to express the unique characteristics of each agent. Besides, to overcome the consistent agent problem of NCC, a split loss is introduced to distinguish different agents and reduce the number of groups. Results reveal that the proposed method makes excellent coordination and achieves a significant improvement in the SMAC benchmark. Due to the group concept, our approach maintains excellent performance with an increase in the number of agents.