Ecological Driving Framework of Hybrid Electric Vehicle Based on Heterogeneous Multi-Agent Deep Reinforcement Learning

Hybrid electric vehicles (HEVs) have great potential to be discovered in terms of energy saving and emission reduction, and ecological driving provides theoretical guidance for giving full play to their advantages in real traffic scenarios. In order to implement an ecological driving strategy with the lowest cost throughout the life cycle in a car-following scenario, the safety and comfort, fuel economy, and battery health need to be considered, which is a complex nonlinear and multiobjective coupled optimization task. Therefore, a novel multi-agent deep deterministic policy gradient (MADDPG) based framework with two heterogeneous agents to optimize adaptive cruise control (ACC) and energy management strategy (EMS), respectively, is proposed, thereby decoupling optimization objectives of different domains. Because of the asynchronous of multi-agents, different learning rate schedules are analyzed to coordinate the learning process to optimize training results; an improvement on the prioritized experience replay (PER) technique is proposed, which improves the optimization performance of the original MADDPG method by more than 10%. Simulations under mixed driving cycles show that, on the premise of ensuring car-following performance, the overall driving cost, including fuel consumption and battery health degradation of the MADDPG-based method, can reach 93.88% of that of DP, and the proposed algorithm has good adaptability to different driving conditions.