Uncovering Interpretable Internal States of Merging Tasks at Highway on-Ramps for Autonomous Driving Decision-Making

Humans make daily routine decisions based on their internal states in intricate interaction scenarios. This article presents a probabilistically reconstructive learning approach to identify the internal states of multivehicle sequential interactions when merging at highway on-ramps. We treated the merging task's sequential decision as a dynamic, stochastic process and then integrated the internal states into a hidden Markov model (HMM)-Gaussian mixture regression (GMR) model, a probabilistic combination of an extended GMR and HMM. We also developed a variant of the expectation-maximization (EM) algorithm to estimate the model parameters and verified it based on a real-world dataset. Experiment results reveal that three interpretable internal states can semantically describe the interactive merge procedure at highway on-ramps. This finding provides a basis for developing an efficient model-based decision-making algorithm for autonomous vehicles (AVs) in a partially observable environment. Note to Practitioners - Model-based learning approaches have obtained increasing attention in decision-making design due to their stability and interpretability. This article was built upon two facts: 1) intelligent agents can only receive partially observable environmental information directly through their equipped sensors in the real world and 2) humans mainly utilize the internal states and associated dynamics inferred from observations to make proper decisions in complex environments. Similarly, autonomous vehicles (AVs) need to understand, infer, anticipate, and exploit the internal states of dynamic environments. Applying probabilistic decision-making models to AVs requires updating the internal states' beliefs and associated dynamics after getting new observations. The designed and verified emission model in hidden Markov model (HMM)-Gaussian mixture regression (GMR) provides a modifiable functional module for online updates of the associated internal states. Experiment results based on the real-world driving dataset demonstrate that the internal states extracted using HMM-GMR can represent the dynamic decision-making process semantically and make an accurate prediction.