Energy Management Strategy Based on an Improved TD3 Reinforcement Algorithm with Novel Experience Replay

Zegong Niu; Ruchen Huang; Hongwen He; Zhiqiang Zhou; Qicong Su

doi:10.1109/VPPC60535.2023.10403251

Energy Management Strategy Based on an Improved TD3 Reinforcement Algorithm with Novel Experience Replay

Zegong Niu, Ruchen Huang, Hongwen He^*, Zhiqiang Zhou, Qicong Su

^*Corresponding author for this work

School of Mechanical Engineering

Beijing Institute of Technology

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

The energy management strategy (EMS) plays an important part in the systematic control of hybrid electric vehicles (HEVs). In recent years, the EMS based on deep reinforcement learning (DRL) receives more attention. This paper proposes an EMS based on TD3 deep reinforcement learning algorithm with novel experience replay. The experience replay is introduced to select samples via an evaluation network aiming to improve the learning ability and the convergence speed. The results show that compared with the traditional TD3-based EMS, the proposed EMS reduces the training time by 8.73% and improves the fuel economy by 2.14%.

Original language	English
Title of host publication	2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9798350344455
DOIs	https://doi.org/10.1109/VPPC60535.2023.10403251
Publication status	Published - 2023
Event	19th IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Milan, Italy Duration: 24 Oct 2023 → 27 Oct 2023

Publication series

Name	2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings

Conference

Conference	19th IEEE Vehicle Power and Propulsion Conference, VPPC 2023
Country/Territory	Italy
City	Milan
Period	24/10/23 → 27/10/23

Keywords

Deep reinforcement learning
Energy management strategy
Hybrid electric bus
TD3

Access to Document

10.1109/VPPC60535.2023.10403251

Cite this

Niu, Z., Huang, R., He, H., Zhou, Z., & Su, Q. (2023). Energy Management Strategy Based on an Improved TD3 Reinforcement Algorithm with Novel Experience Replay. In 2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings (2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/VPPC60535.2023.10403251

Niu, Zegong ; Huang, Ruchen ; He, Hongwen et al. / Energy Management Strategy Based on an Improved TD3 Reinforcement Algorithm with Novel Experience Replay. 2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2023. (2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings).

@inproceedings{89b9ea326d854d72b43e87e3499f20b1,

title = "Energy Management Strategy Based on an Improved TD3 Reinforcement Algorithm with Novel Experience Replay",

abstract = "The energy management strategy (EMS) plays an important part in the systematic control of hybrid electric vehicles (HEVs). In recent years, the EMS based on deep reinforcement learning (DRL) receives more attention. This paper proposes an EMS based on TD3 deep reinforcement learning algorithm with novel experience replay. The experience replay is introduced to select samples via an evaluation network aiming to improve the learning ability and the convergence speed. The results show that compared with the traditional TD3-based EMS, the proposed EMS reduces the training time by 8.73% and improves the fuel economy by 2.14%.",

keywords = "Deep reinforcement learning, Energy management strategy, Hybrid electric bus, TD3",

author = "Zegong Niu and Ruchen Huang and Hongwen He and Zhiqiang Zhou and Qicong Su",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 19th IEEE Vehicle Power and Propulsion Conference, VPPC 2023 ; Conference date: 24-10-2023 Through 27-10-2023",

year = "2023",

doi = "10.1109/VPPC60535.2023.10403251",

language = "English",

series = "2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings",

address = "United States",

}

Niu, Z, Huang, R, He, H, Zhou, Z & Su, Q 2023, Energy Management Strategy Based on an Improved TD3 Reinforcement Algorithm with Novel Experience Replay. in 2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings. 2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings, Institute of Electrical and Electronics Engineers Inc., 19th IEEE Vehicle Power and Propulsion Conference, VPPC 2023, Milan, Italy, 24/10/23. https://doi.org/10.1109/VPPC60535.2023.10403251

Energy Management Strategy Based on an Improved TD3 Reinforcement Algorithm with Novel Experience Replay. / Niu, Zegong; Huang, Ruchen; He, Hongwen et al.
2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2023. (2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Energy Management Strategy Based on an Improved TD3 Reinforcement Algorithm with Novel Experience Replay

AU - Niu, Zegong

AU - Huang, Ruchen

AU - He, Hongwen

AU - Zhou, Zhiqiang

AU - Su, Qicong

PY - 2023

Y1 - 2023

N2 - The energy management strategy (EMS) plays an important part in the systematic control of hybrid electric vehicles (HEVs). In recent years, the EMS based on deep reinforcement learning (DRL) receives more attention. This paper proposes an EMS based on TD3 deep reinforcement learning algorithm with novel experience replay. The experience replay is introduced to select samples via an evaluation network aiming to improve the learning ability and the convergence speed. The results show that compared with the traditional TD3-based EMS, the proposed EMS reduces the training time by 8.73% and improves the fuel economy by 2.14%.

AB - The energy management strategy (EMS) plays an important part in the systematic control of hybrid electric vehicles (HEVs). In recent years, the EMS based on deep reinforcement learning (DRL) receives more attention. This paper proposes an EMS based on TD3 deep reinforcement learning algorithm with novel experience replay. The experience replay is introduced to select samples via an evaluation network aiming to improve the learning ability and the convergence speed. The results show that compared with the traditional TD3-based EMS, the proposed EMS reduces the training time by 8.73% and improves the fuel economy by 2.14%.

KW - Deep reinforcement learning

KW - Energy management strategy

KW - Hybrid electric bus

KW - TD3

UR - http://www.scopus.com/inward/record.url?scp=85185344684&partnerID=8YFLogxK

U2 - 10.1109/VPPC60535.2023.10403251

DO - 10.1109/VPPC60535.2023.10403251

M3 - Conference contribution

AN - SCOPUS:85185344684

T3 - 2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings

BT - 2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 19th IEEE Vehicle Power and Propulsion Conference, VPPC 2023

Y2 - 24 October 2023 through 27 October 2023

ER -

Niu Z, Huang R, He H, Zhou Z, Su Q. Energy Management Strategy Based on an Improved TD3 Reinforcement Algorithm with Novel Experience Replay. In 2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2023. (2023 IEEE Vehicle Power and Propulsion Conference, VPPC 2023 - Proceedings). doi: 10.1109/VPPC60535.2023.10403251

Energy Management Strategy Based on an Improved TD3 Reinforcement Algorithm with Novel Experience Replay

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this