Multi-Vehicle Interaction-Aware Energy Management for Connected Hybrid Electric Vehicles via Deep Reinforcement Learning

Energy management holds the key to en-hancing the energy efficiency of hybrid electric vehicles (HEVs). However, it brings a high level of uncertainty to the driving of HEVs in dense and dynamic traffic environments with multi-vehicle interactions, which consequently influences the performance and adapt-ability of onboard energy management. Concentrated on this issue, this paper proposed a deep reinforcement learning-based energy management method enabled by multi-vehicle interaction awareness. First, oriented to-ward energy management, a feature extraction module is presented to capture and extract vehicle-to-vehicle interactions in real time by the attention mechanism. This module is capable of dealing with time-varying sequences and counts of observed surrounding vehicles over time. Then, it is integrated into the development of parameterized energy management strategies (EMSs), which are optimized by the proximal policy optimization method. The proposed EMS is trained and exam-ined in a connected vehicle environment. Comparative simulation results indicate that it enhances the training stability by leveraging the ego-HEV-centered multi-vehicle interaction features. It significantly narrows the fuel economy gap with the dynamic programming-based benchmark EMS down to about 5.6% from 8.7%. The adaptability validation in test driving scenar-ios, encompassing distinct driving cycles and various initial powertrain states, also exhibits consistent charge-sustaining and energy-saving performances.