TY - JOUR
T1 - A real-time energy management strategy for off-road hybrid electric vehicles based on the expected SARSA
AU - Bo, Lin
AU - Han, Lijin
AU - Xiang, Changle
AU - Liu, Hui
AU - Ma, Tian
N1 - Publisher Copyright:
© IMechE 2022.
PY - 2023/2
Y1 - 2023/2
N2 - To improve the power performance and fuel economy and reduce the online computation time of hybrid electric vehicles (HEVs) in off-road driving environments, a real-time energy management strategy based on the expected state-action-reward-state’-action’ (SARSA) algorithm is proposed. This strategy achieves the control effect through coordinated control of the engine generator set (EGS) and the energy distribution between the battery and the EGS. The driving environment is expressed as a transition probability matrix (TPM) of the electric power demand, and the optimal control problem of energy management is solved offline by the expected SARSA algorithm to obtain the control laws. Then, the Kullback-Leibler (KL) divergence rate is used as the evaluation index of the switching control strategy, and these control laws are used online to realize real-time control. Control strategies based on traditional reinforcement learning (RL), stochastic dynamic programming (SDP), and dynamic programming (DP) are considered benchmark strategies to verify the effectiveness of the proposed energy management strategy based on the expected SARSA. The fuel economy is improved by a maximum of 13.2%, and the elapsed time is reduced by more than 99.8% compared with the values achieved with the benchmark algorithms. These results show that the proposed strategy has flexible adaptability to complex and changeable off-road conditions without prior knowledge of the whole driving cycle.
AB - To improve the power performance and fuel economy and reduce the online computation time of hybrid electric vehicles (HEVs) in off-road driving environments, a real-time energy management strategy based on the expected state-action-reward-state’-action’ (SARSA) algorithm is proposed. This strategy achieves the control effect through coordinated control of the engine generator set (EGS) and the energy distribution between the battery and the EGS. The driving environment is expressed as a transition probability matrix (TPM) of the electric power demand, and the optimal control problem of energy management is solved offline by the expected SARSA algorithm to obtain the control laws. Then, the Kullback-Leibler (KL) divergence rate is used as the evaluation index of the switching control strategy, and these control laws are used online to realize real-time control. Control strategies based on traditional reinforcement learning (RL), stochastic dynamic programming (SDP), and dynamic programming (DP) are considered benchmark strategies to verify the effectiveness of the proposed energy management strategy based on the expected SARSA. The fuel economy is improved by a maximum of 13.2%, and the elapsed time is reduced by more than 99.8% compared with the values achieved with the benchmark algorithms. These results show that the proposed strategy has flexible adaptability to complex and changeable off-road conditions without prior knowledge of the whole driving cycle.
KW - Reinforcement learning
KW - energy management
KW - expected SARSA
KW - hybrid vehicle
UR - http://www.scopus.com/inward/record.url?scp=85125054240&partnerID=8YFLogxK
U2 - 10.1177/09544070221079173
DO - 10.1177/09544070221079173
M3 - Article
AN - SCOPUS:85125054240
SN - 0954-4070
VL - 237
SP - 362
EP - 380
JO - Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering
JF - Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering
IS - 2-3
ER -