A Heuristic Planning Reinforcement Learning-Based Energy Management for Power-Split Plug-in Hybrid Electric Vehicles

This paper proposes a heuristic planning energy management controller, based on a Dyna agent of reinforcement learning (RL) approach, for real-time fuel saving optimization of a plug-in hybrid electric vehicle (PHEV). The presented method is referred to as the Dyna-Η algorithm, which is a model-free online RL algorithm. First, as a case study, a detailed vehicle powertrain modeling of the Chevrolet Volt is built, where all the control components have been experimentally validated. Four traction operation modes are allowed by managing the states of two clutches and one brake. Furthermore, the Dyna-Η algorithm is introduced via incorporating a heuristic planning strategy into a Dyna agent. This is the first time to apply the Dyna-H algorithm in the energy management field of PHEVs. Finally, a comparative analysis of the one-step Q-learning, Dyna, and Dyna-Η algorithms is conducted in simulations. Numerous testing results indicate that the proposed algorithm leads to definite improvements in equivalent fuel economy and computational speed.