Interact, Instruct to Improve: A LLM-Driven Parallel Actor-Reasoner Framework for Enhancing Autonomous Vehicle Interactions

Autonomous Vehicles (AVs) have entered the stage of commercialization, yet their performance in interactive scenarios remains unsatisfactory due to challenges such as decision interpretability, human driver (HV) heterogeneity, and scenario diversity. Recent advances in Large Language Models (LLMs) provide a promising avenue to enhance AV interaction capabilities, but their high computational demand hinders practical deployment. To address these challenges, this paper introduces a parallel Actor-Reasoner framework designed to enable explicit and real-time bidirectional AV-HV interactions. First, the Reasoner employs a localized LLM with CoT reasoning and human instructions to progressively infer HV intent, style, AV action, and eHMI displays during the training stage. During testing, it continues to infer he above information except AV action. The Actor, in turn, is constructed as an interaction memory through the Reasoner's interactions with heterogeneous simulated HVs across diverse scenarios, where the memory partition and two-layer retrieval modules are employed in the construction process. During testing, the Actor is used to retrieve feasible actions for the AV. Ablation studies across multiple scenarios demonstrate that the proposed modules improve interaction success rates by an average of 15% and 12%, respectively. Moreover, comparison studies in multi-vehicle scenarios further show that the proposed Actor-Reasoner framework achieves superior safety while simultaneously improving efficiency. Finally, with the integration of external Human-Machine Interface (eHMI) information derived from the Reasoner's reasoning and feasible actions retrieved from the Actor, the framework is validated in real-world field interactions. Our code is available at https://github.com/FanGShiYuu/Actor-Reasoner.