Reinforcement Learning Improved Control Architecture for Following Vehicle in Platooning Driving

Xiaoran Lu; Yuan Zou; Xudong Zhang; Haitao Liu; Yijie Chen; Bin Zhang

doi:10.1007/978-981-95-4875-0_4

Reinforcement Learning Improved Control Architecture for Following Vehicle in Platooning Driving

Xiaoran Lu
, Yuan Zou^*
, Xudong Zhang
, Haitao Liu
, Yijie Chen
, Bin Zhang

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

To improve the mobility and range of Platooning Driving for material transportation or regional inspection tasks in off-road environments, this paper proposed an reinforcement learning improved architecture for the speed and energy control for the following vehicle, the primary task of the architecture is to fulfill the stable and high maneuverability requirement of the following vehicle, the secondary task is to enhance the fuel economic efficiency of the engine- generator sat.For the speed control system, the Deep Deterministic Policy Gradient (DDPG) algorithm is used to control the speed which is critical for the following task. For the energy management system(EMS), a n-step improved Twin Delayed Deep Deterministic Policy Gradient (TD3-Nstep) algorithm is built to control the engine-generator set for the power supply purpose, instead of power following control method, the EMS aims at stabilizing the bus voltage of the vehicle, which can support the maneuvering characteristics of the platooning driving. Simulation experiments show that the proposed algorithm enables the platooning vehicle to have good performance in high-mobility characteristics and energy system stability.

Original language	English
Title of host publication	Intelligent Vehicles - 3rd CCF Intelligent Vehicles Symposium, CIVS 2025, Revised Selected Papers
Editors	Huiyun Li, Zhongli Wang, Shuai Zhao, Peng Sun, Michael Herrmann, Xi Zheng, Yuling Liu
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	38-50
Number of pages	13
ISBN (Print)	9789819548743
DOIs	https://doi.org/10.1007/978-981-95-4875-0_4
Publication status	Published - 2026
Externally published	Yes
Event	3rd CCF Intelligent Vehicles Symposium, CIVS 2025 - Hangzhou, China Duration: 16 Aug 2025 → 18 Aug 2025

Publication series

Name	Communications in Computer and Information Science
Volume	2631 CCIS
ISSN (Print)	1865-0929
ISSN (Electronic)	1865-0937

Conference

Conference	3rd CCF Intelligent Vehicles Symposium, CIVS 2025
Country/Territory	China
City	Hangzhou
Period	16/08/25 → 18/08/25

Keywords

energy management system
following vehicle
platooning driving
reinforcement learning
speed and energy control

Access to Document

10.1007/978-981-95-4875-0_4

Cite this

Lu, X., Zou, Y., Zhang, X., Liu, H., Chen, Y., & Zhang, B. (2026). Reinforcement Learning Improved Control Architecture for Following Vehicle in Platooning Driving. In H. Li, Z. Wang, S. Zhao, P. Sun, M. Herrmann, X. Zheng, & Y. Liu (Eds.), Intelligent Vehicles - 3rd CCF Intelligent Vehicles Symposium, CIVS 2025, Revised Selected Papers (pp. 38-50). (Communications in Computer and Information Science; Vol. 2631 CCIS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-95-4875-0_4

Lu, Xiaoran ; Zou, Yuan ; Zhang, Xudong et al. / Reinforcement Learning Improved Control Architecture for Following Vehicle in Platooning Driving. Intelligent Vehicles - 3rd CCF Intelligent Vehicles Symposium, CIVS 2025, Revised Selected Papers. editor / Huiyun Li ; Zhongli Wang ; Shuai Zhao ; Peng Sun ; Michael Herrmann ; Xi Zheng ; Yuling Liu. Springer Science and Business Media Deutschland GmbH, 2026. pp. 38-50 (Communications in Computer and Information Science).

@inproceedings{3533f6db0c594757a1b5af19d6eba6eb,

title = "Reinforcement Learning Improved Control Architecture for Following Vehicle in Platooning Driving",

abstract = "To improve the mobility and range of Platooning Driving for material transportation or regional inspection tasks in off-road environments, this paper proposed an reinforcement learning improved architecture for the speed and energy control for the following vehicle, the primary task of the architecture is to fulfill the stable and high maneuverability requirement of the following vehicle, the secondary task is to enhance the fuel economic efficiency of the engine- generator sat.For the speed control system, the Deep Deterministic Policy Gradient (DDPG) algorithm is used to control the speed which is critical for the following task. For the energy management system(EMS), a n-step improved Twin Delayed Deep Deterministic Policy Gradient (TD3-Nstep) algorithm is built to control the engine-generator set for the power supply purpose, instead of power following control method, the EMS aims at stabilizing the bus voltage of the vehicle, which can support the maneuvering characteristics of the platooning driving. Simulation experiments show that the proposed algorithm enables the platooning vehicle to have good performance in high-mobility characteristics and energy system stability.",

keywords = "energy management system, following vehicle, platooning driving, reinforcement learning, speed and energy control",

author = "Xiaoran Lu and Yuan Zou and Xudong Zhang and Haitao Liu and Yijie Chen and Bin Zhang",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2026.; 3rd CCF Intelligent Vehicles Symposium, CIVS 2025 ; Conference date: 16-08-2025 Through 18-08-2025",

year = "2026",

doi = "10.1007/978-981-95-4875-0\_4",

language = "English",

isbn = "9789819548743",

series = "Communications in Computer and Information Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "38--50",

editor = "Huiyun Li and Zhongli Wang and Shuai Zhao and Peng Sun and Michael Herrmann and Xi Zheng and Yuling Liu",

booktitle = "Intelligent Vehicles - 3rd CCF Intelligent Vehicles Symposium, CIVS 2025, Revised Selected Papers",

address = "Germany",

}

Lu, X, Zou, Y, Zhang, X, Liu, H, Chen, Y & Zhang, B 2026, Reinforcement Learning Improved Control Architecture for Following Vehicle in Platooning Driving. in H Li, Z Wang, S Zhao, P Sun, M Herrmann, X Zheng & Y Liu (eds), Intelligent Vehicles - 3rd CCF Intelligent Vehicles Symposium, CIVS 2025, Revised Selected Papers. Communications in Computer and Information Science, vol. 2631 CCIS, Springer Science and Business Media Deutschland GmbH, pp. 38-50, 3rd CCF Intelligent Vehicles Symposium, CIVS 2025, Hangzhou, China, 16/08/25. https://doi.org/10.1007/978-981-95-4875-0_4

Reinforcement Learning Improved Control Architecture for Following Vehicle in Platooning Driving. / Lu, Xiaoran; Zou, Yuan; Zhang, Xudong et al.
Intelligent Vehicles - 3rd CCF Intelligent Vehicles Symposium, CIVS 2025, Revised Selected Papers. ed. / Huiyun Li; Zhongli Wang; Shuai Zhao; Peng Sun; Michael Herrmann; Xi Zheng; Yuling Liu. Springer Science and Business Media Deutschland GmbH, 2026. p. 38-50 (Communications in Computer and Information Science; Vol. 2631 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Reinforcement Learning Improved Control Architecture for Following Vehicle in Platooning Driving

AU - Lu, Xiaoran

AU - Zou, Yuan

AU - Zhang, Xudong

AU - Liu, Haitao

AU - Chen, Yijie

AU - Zhang, Bin

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2026.

PY - 2026

Y1 - 2026

N2 - To improve the mobility and range of Platooning Driving for material transportation or regional inspection tasks in off-road environments, this paper proposed an reinforcement learning improved architecture for the speed and energy control for the following vehicle, the primary task of the architecture is to fulfill the stable and high maneuverability requirement of the following vehicle, the secondary task is to enhance the fuel economic efficiency of the engine- generator sat.For the speed control system, the Deep Deterministic Policy Gradient (DDPG) algorithm is used to control the speed which is critical for the following task. For the energy management system(EMS), a n-step improved Twin Delayed Deep Deterministic Policy Gradient (TD3-Nstep) algorithm is built to control the engine-generator set for the power supply purpose, instead of power following control method, the EMS aims at stabilizing the bus voltage of the vehicle, which can support the maneuvering characteristics of the platooning driving. Simulation experiments show that the proposed algorithm enables the platooning vehicle to have good performance in high-mobility characteristics and energy system stability.

AB - To improve the mobility and range of Platooning Driving for material transportation or regional inspection tasks in off-road environments, this paper proposed an reinforcement learning improved architecture for the speed and energy control for the following vehicle, the primary task of the architecture is to fulfill the stable and high maneuverability requirement of the following vehicle, the secondary task is to enhance the fuel economic efficiency of the engine- generator sat.For the speed control system, the Deep Deterministic Policy Gradient (DDPG) algorithm is used to control the speed which is critical for the following task. For the energy management system(EMS), a n-step improved Twin Delayed Deep Deterministic Policy Gradient (TD3-Nstep) algorithm is built to control the engine-generator set for the power supply purpose, instead of power following control method, the EMS aims at stabilizing the bus voltage of the vehicle, which can support the maneuvering characteristics of the platooning driving. Simulation experiments show that the proposed algorithm enables the platooning vehicle to have good performance in high-mobility characteristics and energy system stability.

KW - energy management system

KW - following vehicle

KW - platooning driving

KW - reinforcement learning

KW - speed and energy control

UR - https://www.scopus.com/pages/publications/105028087549

U2 - 10.1007/978-981-95-4875-0_4

DO - 10.1007/978-981-95-4875-0_4

M3 - Conference contribution

AN - SCOPUS:105028087549

SN - 9789819548743

T3 - Communications in Computer and Information Science

SP - 38

EP - 50

BT - Intelligent Vehicles - 3rd CCF Intelligent Vehicles Symposium, CIVS 2025, Revised Selected Papers

A2 - Li, Huiyun

A2 - Wang, Zhongli

A2 - Zhao, Shuai

A2 - Sun, Peng

A2 - Herrmann, Michael

A2 - Zheng, Xi

A2 - Liu, Yuling

PB - Springer Science and Business Media Deutschland GmbH

T2 - 3rd CCF Intelligent Vehicles Symposium, CIVS 2025

Y2 - 16 August 2025 through 18 August 2025

ER -

Lu X, Zou Y, Zhang X, Liu H, Chen Y, Zhang B. Reinforcement Learning Improved Control Architecture for Following Vehicle in Platooning Driving. In Li H, Wang Z, Zhao S, Sun P, Herrmann M, Zheng X, Liu Y, editors, Intelligent Vehicles - 3rd CCF Intelligent Vehicles Symposium, CIVS 2025, Revised Selected Papers. Springer Science and Business Media Deutschland GmbH. 2026. p. 38-50. (Communications in Computer and Information Science). doi: 10.1007/978-981-95-4875-0_4

Reinforcement Learning Improved Control Architecture for Following Vehicle in Platooning Driving

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this