HIRO: Historical intent reasoning with observation for multi-agent reinforcement learning

Multi-Agent Reinforcement Learning (MARL) has been widely applied to cooperative and competitive multi-agent tasks. However, in partially observable environments, each agent can only access its local observation during execution and cannot directly obtain global state information. This is one of the core causes of the non-stationarity problem in MARL, posing significant challenges for multi-agent cooperative decision-making. To address these issues, this paper proposes HIRO (Historical Intent Reasoning with Observation for Multi-Agent Reinforcement Learning), which infers teammates’ latent intents by observing their historical actions, reconstructs their local observations and integrates the inferred information into the agent's own observations, thereby enabling more efficient cooperation. HIRO consists of two core modules: (1) the Intent Encoder, which encodes teammates’ historical action sequences to infer their current probable intents and further reconstruct their local observations; and (2) the Observation Integration Networks, which fuse the inferred intent information with the agent's own local observations to enrich the observational context. We conduct extensive experiments on two benchmark environments: StarCraft II Multi-Agent Challenge (SMAC) and Google Research Football (GRF). The results demonstrate that HIRO outperforms existing mainstream methods significantly in discernible, complex, and challenging scenarios.