A Visual Attention-Informed Scene Graph Approach for Predicting Risk of Vulnerable Road Users

Accurately predicting the risk of vulnerable road users (VRUs) in advance is critical for enhancing the safety of advanced driver assistance systems. However, most existing approaches rely solely on sensor data and fail to fully leverage driver attention, which is implicitly embedded in their visual behavior. In this paper, a novel multi-object risk prediction approach is proposed to incorporate driver gaze information into a spatio-temporal scene graph convolutional neural network for predicting the risk of VRUs. First, the attention state of each road user is calculated using driver gaze information. Then, attention features are embedded into a data-driven scene graph construction module. Finally, the resulting spatio-temporal scene graphs are processed using graph and temporal convolutions to predict the time to collision for each road user. Experimental results show that, compared with baseline methods, our approach maintains high prediction accuracy even over extended prediction horizons, enabling earlier and more reliable identification of high-risk VRUs in driving environments.