TY - JOUR
T1 - Off-road hybrid electric vehicle energy management strategy using multi-agent soft actor-critic with collaborative-independent algorithm
AU - Liu, Hui
AU - You, Congwen
AU - Han, Lijin
AU - Yang, Ningkang
AU - Liu, Baoshuai
N1 - Publisher Copyright:
© 2025 Elsevier Ltd
PY - 2025/8/1
Y1 - 2025/8/1
N2 - Hybrid electric vehicles (HEVs) reduce carbon emissions and save energy, and hybrid energy storage system (HESS) consist of a battery and a supercapacitor which has high energy density and high power density. The HEV equipped with HESS performs better in off-road conditions than single energy storage source. However, its energy management requires multiple input and multiple output (MIMO) control. In this paper, a multi-agent soft actor-critic (MASAC) based energy management strategy (EMS) is proposed to solve the multi-objective optimizing problem considering fuel economy, maintaining state of charge (SOC) and reducing battery state of health (SOH) decay. MASAC based EMS has two advantages: 1) it decomposed the search space into two subspaces, improving the learning efficiency. 2) a novel collaborative-independent algorithm is proposed to allocate rewards among agents, thereby improving the learning stability. Thus, the optimal actions are efficiently and collaboratively learned by two agents, engine agent and HESS agent, showing better performance in multi-objective optimization. In the simulation, the proposed EMS is compared with dynamic programming (DP) and soft actor-critic (SAC) in both off-road driving cycle and standard driving cycles. Simulation results show that the proposed collaborative-independent algorithm enhances the learning efficiency and learning stability of MASAC, while improving the real-time performance of EMS. In off-road conditions, the equivalent fuel consumption of MASAC is slightly better than that of DP. The SOH decay of MASAC is only 20 % higher than DP, significantly outperforming SAC. Furthermore, MASAC demonstrates superior performance in three standard working cycles when compared with SAC.
AB - Hybrid electric vehicles (HEVs) reduce carbon emissions and save energy, and hybrid energy storage system (HESS) consist of a battery and a supercapacitor which has high energy density and high power density. The HEV equipped with HESS performs better in off-road conditions than single energy storage source. However, its energy management requires multiple input and multiple output (MIMO) control. In this paper, a multi-agent soft actor-critic (MASAC) based energy management strategy (EMS) is proposed to solve the multi-objective optimizing problem considering fuel economy, maintaining state of charge (SOC) and reducing battery state of health (SOH) decay. MASAC based EMS has two advantages: 1) it decomposed the search space into two subspaces, improving the learning efficiency. 2) a novel collaborative-independent algorithm is proposed to allocate rewards among agents, thereby improving the learning stability. Thus, the optimal actions are efficiently and collaboratively learned by two agents, engine agent and HESS agent, showing better performance in multi-objective optimization. In the simulation, the proposed EMS is compared with dynamic programming (DP) and soft actor-critic (SAC) in both off-road driving cycle and standard driving cycles. Simulation results show that the proposed collaborative-independent algorithm enhances the learning efficiency and learning stability of MASAC, while improving the real-time performance of EMS. In off-road conditions, the equivalent fuel consumption of MASAC is slightly better than that of DP. The SOH decay of MASAC is only 20 % higher than DP, significantly outperforming SAC. Furthermore, MASAC demonstrates superior performance in three standard working cycles when compared with SAC.
KW - Energy management
KW - Hybrid energy storage system
KW - Multi-agent reinforcement learning
KW - Off-road hybrid electric vehicle
KW - Soft actor-critic
UR - http://www.scopus.com/inward/record.url?scp=105004800351&partnerID=8YFLogxK
U2 - 10.1016/j.energy.2025.136463
DO - 10.1016/j.energy.2025.136463
M3 - Article
AN - SCOPUS:105004800351
SN - 0360-5442
VL - 328
JO - Energy
JF - Energy
M1 - 136463
ER -