Heuristic Energy Management Strategy of Hybrid Electric Vehicle Based on Deep Reinforcement Learning with Accelerated Gradient Optimization

In this article, a heuristic deep reinforcement learning (DRL) control strategy is proposed for the energy management of the series hybrid electric vehicle (SHEV). First, the powertrain model of the vehicle and the formulas of the energy management strategy (EMS) are introduced. Then, the complete control framework with a nested loop logic is constructed for the EMS. In this control framework, the heuristic experience replay (HER) is proposed to achieve more reasonable experience sampling and improve training efficiency. Besides, the adaptive moment estimation optimization method with the Nesterov accelerated gradient called NAG-Adam is presented to achieve a better optimization effect. Subsequently, the performance of the proposed control strategy is verified by the high-precision driving cycle. The simulation results show that the newly proposed method can achieve faster training speed and higher fuel economy compared to the existing DRL methods and is close to the global optimum. Finally, the adaptability, stability, and robustness of the proposed method are verified by applying different driving cycles.