Formulating and Unveiling In-context Learning over Graph Tasks

In-context learning (ICL) enables large language models (LLMs) to adapt to new tasks by conditioning on demonstrations, without updating model parameters. While recent studies have examined the ICL capabilities of LLMs over graph tasks, the mechanism by which demonstrations influence model behavior remains difficult to formalize and poorly understood. In this paper, we present thorough formulations, innovative mechanisms, and comprehensive benchmarks for investigating ICL over graph tasks. We introduce the first unified formulation that explicitly models demonstration number, graph structure, and task category over graph tasks, enabling independent variation of these factors and systematic analysis of their effects. We unveil how demonstrations in graph tasks activate the ICL capabilities of LLMs, showing that LLMs adjust their predictions by weighting and aggregating the representations of the query and demonstrations. We introduce two new benchmarks designed to enable rigorous evaluation under controlled structural and task variations, comprising a total of 17,155 graph questions across graphs of varying sizes and multiple task categories. Using these benchmarks, our experiments explore how demonstrations activate the ICL capabilities of LLMs over graph tasks from the perspectives of demonstration number, graph structure, and task category, and provide empirical support for the proposed formulations and mechanisms. The benchmarks and code are available at: https://github.com/zhangchunhui2024/Graph-ICL .