An real-time intelligent energy management based on deep reinforcement learning and model predictive control for hybrid electric vehicles considering battery life

To alleviate environmental pollution and energy crisis, the large-scale deployment of hybrid electric vehicles (HEVs) is a promising solution and their energy management is a critical technology for enhancing the fuel efficiency. This paper proposes a real-time energy management strategy (EMS) for HEVs that integrates model predictive control (MPC) with twin delayed deep deterministic policy gradient(TD3) to improve fuel economy and minimize battery degradation. First, considering the dynamic actual driving conditions, an online recursive high-order Markov Chain(MC) model is developed to predict the randomness of the environment in the MPC framework, an EMS controller is then developed based on the advanced TD3 algorithm to generate reliable State of Charge (SOC) reference sequences and action reference sequences. moreover, an improved Sequential Quadratic Programming (SQP) algorithm is devised to solve the MPC problem for enhancing real-time performance. Meanwhile, coordinated control algorithms on the dynamic conditions of the system is designed to incorporate the response characteristics of key system components into the energy management problem. Then, the DP, MPC-RL and Rule-based strategies are designed as baselines to compare with the proposed strategy under three unknown driving cycles. The results demonstrates satisfactory performance in fuel economy, real-time performance, robustness and reduction of battery life loss. Finally, a hardware-in-the-loop(HIL) experiment validates its practical applicability.