Human-Guided Reinforcement Learning Using Multi Q-Advantage for End-to-End Autonomous Driving

Pei Wang, Yong Wang, Hongwen He*, Jingda Wu, Zirui Kuang

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Reinforcement learning (RL) is a promising approach for end-to-end autonomous driving. However, training a RL strategy for autonomous driving is challenging, requiring a meticulously crafted reward function and an extensive number of interactions. This necessitates reconsidering the integration of RL with human guidance, acknowledging the potential benefits of combining machine learning with human expertise. Herein, we propose a novel human-guided reinforcement learning (HGRL) algorithm for policy training in the context of end-to-end autonomous driving. Our HGRL method employs several mechanisms to enhance the effectiveness of low-intensity human guidance, including a regressed human policy model, multiple Q-advantage technique, and prioritized human experience replay. The proposed method is evaluated on a challenging lane-changing and overtaking driving task based only on small neural networks and image inputs. Simulation results show that our method surpasses current state-of-the-art human-guided RL algorithms in both driving performance and generalization performance. Furthermore, the proposed HGRL method trained in simulation is then transferred to a real-world autonomous vehicle.

Original languageEnglish
Title of host publicationProceedings of the 2024 8th CAA International Conference on Vehicular Control and Intelligence, CVCI 2024
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9798331504892
DOIs
Publication statusPublished - 2024
Event8th CAA International Conference on Vehicular Control and Intelligence, CVCI 2024 - Chongqing, China
Duration: 25 Oct 202427 Oct 2024

Publication series

NameProceedings of the 2024 8th CAA International Conference on Vehicular Control and Intelligence, CVCI 2024

Conference

Conference8th CAA International Conference on Vehicular Control and Intelligence, CVCI 2024
Country/TerritoryChina
CityChongqing
Period25/10/2427/10/24

Keywords

  • au-tonomous driving
  • human guidance
  • Q-advantage integration
  • reinforcement learning
  • sim-to-real transfer

Fingerprint

Dive into the research topics of 'Human-Guided Reinforcement Learning Using Multi Q-Advantage for End-to-End Autonomous Driving'. Together they form a unique fingerprint.

Cite this