Towards sustainable and intelligent urban transportation: A novel deep transfer reinforcement learning framework for eco-driving of fuel cell buses

Eco-driving is a sustainable technology that optimizes both energy management and speed planning for electrified vehicles. Particularly when combined with emerging deep reinforcement learning (DRL) techniques, eco-driving strategies (EDSs) can be more intelligent. However, current research on eco-driving, namely the holistic solution, lags behind the advancements in its sub-problem namely energy management, and the development of DRL-based EDSs remains time-consuming. Since energy management is a sub-task of eco-driving, it offers a potential way to rapidly develop EDSs by reusing pre-trained energy management strategies (EMSs). Based on this, this paper proposes an expedited method for developing soft actor-critic (SAC) based EDSs for fuel cell buses (FCBs) in the vehicle-following scenario. To ensure that SAC-based EMSs can be effectively transferred to EDSs, an innovative heterogeneous deep transfer reinforcement learning framework is designed. Within this framework, all the knowledge learned in the source EMS can be transferred and reused by the target EDS. More importantly, the transferability of heterogeneous deep neural networks and heterogeneous experience replay buffers is particularly verified. Simulation results show that the proposed framework provides a 71.01 % acceleration in convergence speed and a 7.30 % improvement in fuel economy. This article contributes to correlating different optimization tasks of FCBs through advanced artificial intelligence technologies.