Expert-demonstration-augmented reinforcement learning for lane-change-aware eco-driving traversing consecutive traffic lights

Chuntao Zhang, Wenhui Huang, Xingyu Zhou, Chen Lv, Chao Sun*

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

5 Citations (Scopus)

Abstract

Eco-driving methods incorporating lateral motion exhibit enhanced energy-saving prospects in multi-lane traffic contexts, yet the randomly distributed obstructing vehicles and sparse traffic lights pose challenges in assessing the long-term value of instantaneous actions, impeding further improvement in energy efficiency. In response to this issue, a deep reinforcement learning (DRL)-based eco-driving method is proposed and augmented with the expert demonstration mechanism. Specifically, a Markov decision process matching with the target eco-driving scenario is systematically constructed, with which, the formulated DRL algorithm, parametrized soft actor-critic (PSAC), is trained to realize the integrated optimization of speed planning and lane-changing maneuver. To promote the training performance of PSAC under sparse rewards concerning traffic lights, an expert eco-driving model and an adaptive sampling approach are incorporated to constitute the expert demonstration mechanism. Simulation results highlight the superior performance of the proposed DRL-based eco-driving method and its training mechanism. Compared with the performance of the PSAC with a pure exploration-based training mechanism, the expert demonstration mechanism promotes the training efficiency and cumulated rewards of PSAC by about 60 % and 21.89 % respectively in the training phase, while in the test phase, a further reduction of 4.23 % benchmarked on a rule-based method is achieved in fuel consumption.

Original languageEnglish
Article number129472
JournalEnergy
Volume286
DOIs
Publication statusPublished - 1 Jan 2024

Keywords

  • Eco-driving
  • Energy economy
  • Expert demonstration
  • Reinforcement learning

Fingerprint

Dive into the research topics of 'Expert-demonstration-augmented reinforcement learning for lane-change-aware eco-driving traversing consecutive traffic lights'. Together they form a unique fingerprint.

Cite this