Hypergraph-Enhanced Multi-Agent Reinforcement Learning for Tiered Mixed Autonomy

Tiered mixed autonomy (TMA) requires connected and autonomous vehicles (CAVs) of varying intelligence levels to coordinate with stochastic human-driven vehicles (HVs), presenting a core challenge for intelligent transportation. Graph-based methods model these interactions through pairwise edges, which cannot capture the group dynamics of vehicle clusters or differentiate cooperation from competition. Hypergraph methods overcome this pairwise constraint but treat all hyperedges uniformly, offering no semantic distinction among interaction types and no mechanism to handle the heterogeneous observation noise critical to decision-making. This article proposes feature-enhanced cluster-based traffic hypergraph (FECTH)-robust Q-mixing networks (RQMIX) to address both limitations jointly. A FECTH constructs three semantically distinct hyperedge types with specialized attribute vectors for intra-cluster cooperation, inter-cluster competition, and human–machine interaction. A RQMIX processes these representations through a gated hypergraph attention network and a dynamic trust mechanism that weights cluster contributions by observation reliability. Experiments achieve 10.7% and 12.3% reductions in task completion time on highway ramps and unsignalized roundabouts, respectively. Hardware-in-the-loop validation confirms robust sim-to-real transferability.