TY - JOUR
T1 - Cooperative traffic optimization with multi-agent reinforcement learning and evolutionary strategy
T2 - Bridging the gap between micro and macro traffic control
AU - Feng, Jianshuai
AU - Lin, Kaize
AU - Shi, Tianyu
AU - Wu, Yuankai
AU - Wang, Yong
AU - Zhang, Hailong
AU - Tan, Huachun
N1 - Publisher Copyright:
© 2024 Elsevier B.V.
PY - 2024/8/1
Y1 - 2024/8/1
N2 - The emergence of connected and autonomous vehicles (CAVs) holds promise for fine-grained traffic control. However, due to the longevity of future mixed traffic scenarios, there is a need for an in-depth exploration of integrating the microscopic speed control of CAVs with the macroscopic variable speed limit (VSL) of human-driven vehicles (HDVs). This paper proposes a Cooperative Traffic Optimization with Multi-agent Reinforcement Learning and Evolutionary VSL (CTO-ME) framework, which combines microscopic CAV control with macroscopic VSL control. The framework incorporates a Graph Attention Mechanism (GATs) into the multi-agent reinforcement learning framework for intelligent decision-making by microscopic-level vehicles. Additionally, an evolutionary strategy is developed to design the VSL network architecture, enabling macroscopic level real-time speed limit adjustments based on infrastructure. A multi-objective reward function is proposed to optimize both micro and macro efficiency and safety, accounting for both vehicle behavior and traffic flow. Experiments on the designed Bottleneck traffic scenarios show that the proposed approach, CTO-ME, is able to achieve superior performance and outperforms other baselines in terms of traffic throughput, average speed, and safety. Specifically, CTO-ME enhances average velocity by 37%, increases overall throughput by 309%, and raises arrival ratio by 70% than traditional Intelligent Driver Model (IDM).
AB - The emergence of connected and autonomous vehicles (CAVs) holds promise for fine-grained traffic control. However, due to the longevity of future mixed traffic scenarios, there is a need for an in-depth exploration of integrating the microscopic speed control of CAVs with the macroscopic variable speed limit (VSL) of human-driven vehicles (HDVs). This paper proposes a Cooperative Traffic Optimization with Multi-agent Reinforcement Learning and Evolutionary VSL (CTO-ME) framework, which combines microscopic CAV control with macroscopic VSL control. The framework incorporates a Graph Attention Mechanism (GATs) into the multi-agent reinforcement learning framework for intelligent decision-making by microscopic-level vehicles. Additionally, an evolutionary strategy is developed to design the VSL network architecture, enabling macroscopic level real-time speed limit adjustments based on infrastructure. A multi-objective reward function is proposed to optimize both micro and macro efficiency and safety, accounting for both vehicle behavior and traffic flow. Experiments on the designed Bottleneck traffic scenarios show that the proposed approach, CTO-ME, is able to achieve superior performance and outperforms other baselines in terms of traffic throughput, average speed, and safety. Specifically, CTO-ME enhances average velocity by 37%, increases overall throughput by 309%, and raises arrival ratio by 70% than traditional Intelligent Driver Model (IDM).
KW - Connected and automated vehicles
KW - Evolution strategy
KW - Graph neural network
KW - Multi-agent reinforcement learning
KW - Variable speed limit
UR - http://www.scopus.com/inward/record.url?scp=85196672537&partnerID=8YFLogxK
U2 - 10.1016/j.physa.2024.129734
DO - 10.1016/j.physa.2024.129734
M3 - Article
AN - SCOPUS:85196672537
SN - 0378-4371
VL - 647
JO - Physica A: Statistical Mechanics and its Applications
JF - Physica A: Statistical Mechanics and its Applications
M1 - 129734
ER -