Heterogeneous multi-agent deep reinforcement learning for eco-driving of hybrid electric tracked vehicles: A heuristic training framework

The prosperity of connected and autonomous vehicle (CAV) technology, as well as artificial intelligence (AI), has enhanced the energy conservation potential of hybrid electric vehicles (HEVs) through deep reinforcement learning (DRL) algorithms. Based on this premise, this article proposes a novel eco-driving strategy that integrates adaptive cruise control (ACC) and energy management strategy (EMS) for a series hybrid electric tracked vehicle (SHETV) based on a multi-agent DRL (MADRL) algorithm. To begin, a heterogeneous multi-agent deep deterministic policy gradient (MADDPG) algorithm is formulated to realize the multi-objective optimization in vehicle-following and energy conservation. Furthermore, a heuristic training framework is designed by incorporating curriculum learning and expert-assisted training methods to improve the training efficiency of DRL agents. Finally, the effectiveness and adaptability of the proposed strategy are validated. Simulation results demonstrate that the heuristic training framework accelerates the convergence speed of MADDPG by 48.59%. Moreover, the proposed strategy outperforms the baseline strategy based on DDPG, achieving a 7.97% improvement in fuel economy while maintaining safe and comfortable vehicle-following performance. This article contributes to energy conservation for a hybrid electric tracked vehicle in real-world traffic scenarios through advanced MADRL methods.