Integrated thermal-energy management for electric vehicles in high-temperature conditions using hierarchical reinforcement learning

The integrated thermal management system (TMS) expends a considerable quantity of energy in high-temperature environments to maintain the battery and motor systems within a reasonable temperature range, while also affecting the overall vehicle's safety and stable operation. In light of these considerations, a hierarchical reinforcement learning approach for integrated TMS is proposed. Given the low-frequency operation of the thermal system, the study employs the Deep Deterministic Policy Gradient (DDPG) with experience replay technical (DDPG-E) to investigate the battery-motor TMS control strategy under car-following behavior. This approach transforms the trade-off between high-frequency planning and low-frequency execution into a Markov Decision Process (MDP), thereby enabling coordinated optimization of multiple objectives. Specifically, in the pre-optimization layer, based on DDPG-E the heat generation optimization (HGO-DDPG-E) is introduced as a multi-objective optimization criterion to achieve “active load reduction” for the battery-motor system. Subsequently, the battery-motor temperature difference and energy consumption of TMS ancillary components are employed as constraints at the integrated control layer for all TMS components, based on the pre-optimization results. The results of the simulation demonstrate that the proposed method achieves an optimization of 15.3% in heat generation and a 14.1% reduction in TMS energy consumption.