Graph Convolution Reinforcement Learning for Decision-Making in Highway Overtaking Scenario

Meng Xiaoqiang; Yang Fan; Li Xueyuan; Liu Qi; Gao Xin; Li Zirui

doi:10.1109/ICIEA54703.2022.10006015

Graph Convolution Reinforcement Learning for Decision-Making in Highway Overtaking Scenario

Meng Xiaoqiang^*, Yang Fan, Li Xueyuan, Liu Qi, Gao Xin, Li Zirui

^*Corresponding author for this work

School of Mechanical Engineering

Beijing Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Citations (Scopus)

Abstract

Overtaking of autonomous vehicles (AVs) is an extremely complex process, which involves many factors and poses great safety hazards. However, most of the current research does not consider the impact of the dynamic environment on autonomous vehicles. In order to solve the multi-agent overtaking problem on the highway, this paper proposes a decision-making algorithm for AVs. The algorithm is based on graph neural network (GNN) and deep reinforcement learning (DRL), and adopts different training methods including as deep Q network (DQN), double DQN, dueling DQN, and D3QN for simulation. Firstly, the simulation environment is a 3-lane highway constructed in sumo. Secondly, there are both human-driven vehicles (HDVs) and AVs with maximum speeds of 10km/h and 20km/h on the highway. Finally, these two kinds of vehicles will appear in the right lane with different probabilities. The training effect is evaluated by the time it takes for the vehicle to enter and exit the current environment and the average speed of the AV. The simulation results show that the algorithm improves the efficiency of the overtaking process and reduces the accident rate.

Original language	English
Title of host publication	ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications
Editors	Wenxiang Xie, Shibin Gao, Xiaoqiong He, Xing Zhu, Jingjing Huang, Weirong Chen, Lei Ma, Haiyan Shu, Wenping Cao, Lijun Jiang, Zeliang Shu
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	417-422
Number of pages	6
ISBN (Electronic)	9781665409841
DOIs	https://doi.org/10.1109/ICIEA54703.2022.10006015
Publication status	Published - 2022
Event	17th IEEE Conference on Industrial Electronics and Applications, ICIEA 2022 - Chengdu, China Duration: 16 Dec 2022 → 19 Dec 2022

Publication series

Name	ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications

Conference

Conference	17th IEEE Conference on Industrial Electronics and Applications, ICIEA 2022
Country/Territory	China
City	Chengdu
Period	16/12/22 → 19/12/22

Keywords

autonomous vehicles
decision-making
deep reinforcement learning
graph neural network
multi-agent

Access to Document

10.1109/ICIEA54703.2022.10006015

Cite this

Xiaoqiang, M., Fan, Y., Xueyuan, L., Qi, L., Xin, G., & Zirui, L. (2022). Graph Convolution Reinforcement Learning for Decision-Making in Highway Overtaking Scenario. In W. Xie, S. Gao, X. He, X. Zhu, J. Huang, W. Chen, L. Ma, H. Shu, W. Cao, L. Jiang, & Z. Shu (Eds.), ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications (pp. 417-422). (ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICIEA54703.2022.10006015

Xiaoqiang, Meng ; Fan, Yang ; Xueyuan, Li et al. / Graph Convolution Reinforcement Learning for Decision-Making in Highway Overtaking Scenario. ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications. editor / Wenxiang Xie ; Shibin Gao ; Xiaoqiong He ; Xing Zhu ; Jingjing Huang ; Weirong Chen ; Lei Ma ; Haiyan Shu ; Wenping Cao ; Lijun Jiang ; Zeliang Shu. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 417-422 (ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications).

@inproceedings{7f7a422d73274bec8bde12cb4538ef74,

title = "Graph Convolution Reinforcement Learning for Decision-Making in Highway Overtaking Scenario",

abstract = "Overtaking of autonomous vehicles (AVs) is an extremely complex process, which involves many factors and poses great safety hazards. However, most of the current research does not consider the impact of the dynamic environment on autonomous vehicles. In order to solve the multi-agent overtaking problem on the highway, this paper proposes a decision-making algorithm for AVs. The algorithm is based on graph neural network (GNN) and deep reinforcement learning (DRL), and adopts different training methods including as deep Q network (DQN), double DQN, dueling DQN, and D3QN for simulation. Firstly, the simulation environment is a 3-lane highway constructed in sumo. Secondly, there are both human-driven vehicles (HDVs) and AVs with maximum speeds of 10km/h and 20km/h on the highway. Finally, these two kinds of vehicles will appear in the right lane with different probabilities. The training effect is evaluated by the time it takes for the vehicle to enter and exit the current environment and the average speed of the AV. The simulation results show that the algorithm improves the efficiency of the overtaking process and reduces the accident rate.",

keywords = "autonomous vehicles, decision-making, deep reinforcement learning, graph neural network, multi-agent",

author = "Meng Xiaoqiang and Yang Fan and Li Xueyuan and Liu Qi and Gao Xin and Li Zirui",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 17th IEEE Conference on Industrial Electronics and Applications, ICIEA 2022 ; Conference date: 16-12-2022 Through 19-12-2022",

year = "2022",

doi = "10.1109/ICIEA54703.2022.10006015",

language = "English",

series = "ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "417--422",

editor = "Wenxiang Xie and Shibin Gao and Xiaoqiong He and Xing Zhu and Jingjing Huang and Weirong Chen and Lei Ma and Haiyan Shu and Wenping Cao and Lijun Jiang and Zeliang Shu",

booktitle = "ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications",

address = "United States",

}

Xiaoqiang, M, Fan, Y, Xueyuan, L, Qi, L, Xin, G & Zirui, L 2022, Graph Convolution Reinforcement Learning for Decision-Making in Highway Overtaking Scenario. in W Xie, S Gao, X He, X Zhu, J Huang, W Chen, L Ma, H Shu, W Cao, L Jiang & Z Shu (eds), ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications. ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications, Institute of Electrical and Electronics Engineers Inc., pp. 417-422, 17th IEEE Conference on Industrial Electronics and Applications, ICIEA 2022, Chengdu, China, 16/12/22. https://doi.org/10.1109/ICIEA54703.2022.10006015

Graph Convolution Reinforcement Learning for Decision-Making in Highway Overtaking Scenario. / Xiaoqiang, Meng; Fan, Yang; Xueyuan, Li et al.
ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications. ed. / Wenxiang Xie; Shibin Gao; Xiaoqiong He; Xing Zhu; Jingjing Huang; Weirong Chen; Lei Ma; Haiyan Shu; Wenping Cao; Lijun Jiang; Zeliang Shu. Institute of Electrical and Electronics Engineers Inc., 2022. p. 417-422 (ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Graph Convolution Reinforcement Learning for Decision-Making in Highway Overtaking Scenario

AU - Xiaoqiang, Meng

AU - Fan, Yang

AU - Xueyuan, Li

AU - Qi, Liu

AU - Xin, Gao

AU - Zirui, Li

PY - 2022

Y1 - 2022

N2 - Overtaking of autonomous vehicles (AVs) is an extremely complex process, which involves many factors and poses great safety hazards. However, most of the current research does not consider the impact of the dynamic environment on autonomous vehicles. In order to solve the multi-agent overtaking problem on the highway, this paper proposes a decision-making algorithm for AVs. The algorithm is based on graph neural network (GNN) and deep reinforcement learning (DRL), and adopts different training methods including as deep Q network (DQN), double DQN, dueling DQN, and D3QN for simulation. Firstly, the simulation environment is a 3-lane highway constructed in sumo. Secondly, there are both human-driven vehicles (HDVs) and AVs with maximum speeds of 10km/h and 20km/h on the highway. Finally, these two kinds of vehicles will appear in the right lane with different probabilities. The training effect is evaluated by the time it takes for the vehicle to enter and exit the current environment and the average speed of the AV. The simulation results show that the algorithm improves the efficiency of the overtaking process and reduces the accident rate.

AB - Overtaking of autonomous vehicles (AVs) is an extremely complex process, which involves many factors and poses great safety hazards. However, most of the current research does not consider the impact of the dynamic environment on autonomous vehicles. In order to solve the multi-agent overtaking problem on the highway, this paper proposes a decision-making algorithm for AVs. The algorithm is based on graph neural network (GNN) and deep reinforcement learning (DRL), and adopts different training methods including as deep Q network (DQN), double DQN, dueling DQN, and D3QN for simulation. Firstly, the simulation environment is a 3-lane highway constructed in sumo. Secondly, there are both human-driven vehicles (HDVs) and AVs with maximum speeds of 10km/h and 20km/h on the highway. Finally, these two kinds of vehicles will appear in the right lane with different probabilities. The training effect is evaluated by the time it takes for the vehicle to enter and exit the current environment and the average speed of the AV. The simulation results show that the algorithm improves the efficiency of the overtaking process and reduces the accident rate.

KW - autonomous vehicles

KW - decision-making

KW - deep reinforcement learning

KW - graph neural network

KW - multi-agent

UR - http://www.scopus.com/inward/record.url?scp=85146879606&partnerID=8YFLogxK

U2 - 10.1109/ICIEA54703.2022.10006015

DO - 10.1109/ICIEA54703.2022.10006015

M3 - Conference contribution

AN - SCOPUS:85146879606

T3 - ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications

SP - 417

EP - 422

BT - ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications

A2 - Xie, Wenxiang

A2 - Gao, Shibin

A2 - He, Xiaoqiong

A2 - Zhu, Xing

A2 - Huang, Jingjing

A2 - Chen, Weirong

A2 - Ma, Lei

A2 - Shu, Haiyan

A2 - Cao, Wenping

A2 - Jiang, Lijun

A2 - Shu, Zeliang

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 17th IEEE Conference on Industrial Electronics and Applications, ICIEA 2022

Y2 - 16 December 2022 through 19 December 2022

ER -

Xiaoqiang M, Fan Y, Xueyuan L, Qi L, Xin G, Zirui L. Graph Convolution Reinforcement Learning for Decision-Making in Highway Overtaking Scenario. In Xie W, Gao S, He X, Zhu X, Huang J, Chen W, Ma L, Shu H, Cao W, Jiang L, Shu Z, editors, ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications. Institute of Electrical and Electronics Engineers Inc. 2022. p. 417-422. (ICIEA 2022 - Proceedings of the 17th IEEE Conference on Industrial Electronics and Applications). doi: 10.1109/ICIEA54703.2022.10006015

Graph Convolution Reinforcement Learning for Decision-Making in Highway Overtaking Scenario

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this