Distributed reinforcement learning with Transformer-based agent for energy optimization with battery health and safety awareness in hybrid electric vehicles

Developing hybrid electric vehicles (HEVs) is helpful for energy saving and emission reduction during the promotion of transportation electrification. Energy management strategy (EMS) is the key to fully exploit the efficiency potential of HEVs. However, current EMS in HEVs faces three major challenges: inadequate simultaneous consideration of battery health and safety, poor feature extraction from existing driving data, and prolonged training time. To address these issues, this paper proposes a novel health- and safety-conscious EMS for HEVs, utilizing a distributed reinforcement learning framework with a Transformer-based agent. The proposed method quantifies battery thermal risks and incorporates degradation costs into the cost function for optimal power allocation. The Transformer network is employed to capture long-range dependencies in driving data. Additionally, a distributed computational framework is implemented to accelerate the training process. Validation through real-world vehicle deployment shows a 97.4% optimality rate, along with significant reductions in both battery temperature and degradation. The distributed framework also reduces training time by over 75%.