TY - JOUR
T1 - Hyper-relational interaction modeling in multi-modal trajectory prediction for intelligent connected vehicles in smart cites
AU - Lu, Yuhuan
AU - Wang, Wei
AU - Bai, Rufan
AU - Zhou, Shengwei
AU - Garg, Lalit
AU - Bashir, Ali Kashif
AU - Jiang, Weiwei
AU - Hu, Xiping
N1 - Publisher Copyright:
© 2024
PY - 2025/2
Y1 - 2025/2
N2 - Trajectory prediction of surrounding traffic participants is vital for the driving safety of Intelligent Connected Vehicles (ICVs). It has been enabled with the help of the availability of multi-sensor information collected by ICVs. For accurately predicting the future movements of traffic agents, it is crucial to subtlety model the inter-agent interaction. However, existing works focus on the correlations between agents and the map information while neglecting the importance of directly modeling the impact of map elements on inter-agent interactions, the direct modeling of which is beneficial for the representation of agent behaviors. Against this background, we propose to model the hyper-relational interaction, which incorporates map elements into the inter-agent interaction. To tackle the hyper-relational interaction, we propose a novel Hyper-relational Multi-modal Trajectory Prediction (HyperMTP) approach. Specifically, a hyper-relational driving graph is first constructed and the hyper-relational interaction is represented as the hyperedge, directly connecting to various nodes (i.e., agents and map elements). Then a structure-aware embedding initialization technique is developed to obtain unbiased initial embeddings. Afterward, hypergraph dual-attention networks are designed to capture correlations between graph elements while retaining the hyper-relational structure. Finally, a heterogeneous Transformer is devised to further capture the correlations between agents’ states and their corresponding hyper-relational interactions. Experimental results show that HyperMTP consistently outperforms the best-performing baseline with an average improvement of 4.8% across two real-world datasets. Moreover, HyperMTP also boosts the interpretability of trajectory prediction by quantifying the impact of map elements on inter-agent interactions.
AB - Trajectory prediction of surrounding traffic participants is vital for the driving safety of Intelligent Connected Vehicles (ICVs). It has been enabled with the help of the availability of multi-sensor information collected by ICVs. For accurately predicting the future movements of traffic agents, it is crucial to subtlety model the inter-agent interaction. However, existing works focus on the correlations between agents and the map information while neglecting the importance of directly modeling the impact of map elements on inter-agent interactions, the direct modeling of which is beneficial for the representation of agent behaviors. Against this background, we propose to model the hyper-relational interaction, which incorporates map elements into the inter-agent interaction. To tackle the hyper-relational interaction, we propose a novel Hyper-relational Multi-modal Trajectory Prediction (HyperMTP) approach. Specifically, a hyper-relational driving graph is first constructed and the hyper-relational interaction is represented as the hyperedge, directly connecting to various nodes (i.e., agents and map elements). Then a structure-aware embedding initialization technique is developed to obtain unbiased initial embeddings. Afterward, hypergraph dual-attention networks are designed to capture correlations between graph elements while retaining the hyper-relational structure. Finally, a heterogeneous Transformer is devised to further capture the correlations between agents’ states and their corresponding hyper-relational interactions. Experimental results show that HyperMTP consistently outperforms the best-performing baseline with an average improvement of 4.8% across two real-world datasets. Moreover, HyperMTP also boosts the interpretability of trajectory prediction by quantifying the impact of map elements on inter-agent interactions.
KW - Hypergraph attention networks
KW - Intelligent connected vehicles
KW - Interaction modeling
KW - Multi-modal trajectory prediction
UR - http://www.scopus.com/inward/record.url?scp=85203493944&partnerID=8YFLogxK
U2 - 10.1016/j.inffus.2024.102682
DO - 10.1016/j.inffus.2024.102682
M3 - Article
AN - SCOPUS:85203493944
SN - 1566-2535
VL - 114
JO - Information Fusion
JF - Information Fusion
M1 - 102682
ER -