Look one step ahead through first-order aggregation in reinforcement learning-based knowledge graph reasoning

Multi-hop reasoning is an effective and interpretable approach for query answering, as it finds reasoning paths over knowledge graphs (KGs) to enhance interpretability. Recent studies have applied reinforcement learning-based (RL-based) methods with policy agents to solve this problem. However, these methods primarily focus on sequential reasoning paths and candidate nodes, while overlooking the local structural information of adjacency subgraphs and the semantic correlations between relations at each decision step. In this paper, we propose a novel RL-based multi-hop reasoning model, Look One Step Ahead (LOSA), which leverages first-order aggregation to regard more expressive adjacency subgraphs rather than nodes as candidate actions and pays attention to the semantic correlations between relations. Specifically, an adjacency aggregation module encodes the local subgraph information and feeds the representations into the policy network to guide decision-making, thereby reducing backtracking and mitigating sparse-reward issues. Furthermore, a semantic matching module is designed to emphasize the semantic correlations of relations to improve the rationality of the reasoning paths. Extensive experiments on benchmark datasets demonstrate the effectiveness of the proposed approach.