基于分层强化学习和社会偏好的自主超车决策系统

To describe the interaction between the host vehicle (HV) and the overtaken vehicle (OV) in overtaking scenarios, the psychological term 'social preference' was introduced to describe the longitudinal behavioral pattern of OV, and a data-driven classification method was adopted to extract the social preference and incorporate it into the design of reinforcement learning based autonomous overtaking decision-making system (RL-based AODMS). By analyzing social preferences of overtaken vehicles based on the realistic overtaking data, this method was able to generate proper overtaking decisions in response to different preferences. First, the state transition probability of an overtaken vehicle during the overtaking interaction was calculated from a large number of realistic overtaking data and divided into three types: altruistic, egoistic, and prosocial. Then, a semi-model-based advanced Q-learning algorithm was proposed to integrate three preferences into decision model training. Meanwhile, an online classifier of social preference was built to determine the real-time preference of the overtaken vehicle. Combined with our previous study on lane-changing controllers, a hierarchical reinforcement learning based autonomous overtaking system (HRL-based AOS) was constructed. Finally, the joint validation on autonomous overtaking was done by collected realistic data and simulation. The results showed that the AODMS considering social preferences can predict the social preference of OV in real time and make reasonable decisions in complicated overtaking scenarios. Meanwhile, compared to the traditional AOS without considering social preference, the complete AOS constructed in this study showed better comfort and stability. To conclude, this study innovatively operationalizes data-driven social preference in overtaking decision making and improving the adaptability and rationality of decisions, which will contribute to the development of safe and reliable AOS.