Real-time adaptive energy management for off-road hybrid electric vehicles based on decision-time planning

Unknown and changeable driving conditions of off-road hybrid electric vehicle (HEV) challenge its energy management strategy (EMS). To tackle this issue, the paper develops a real-time adaptive strategy for off-road HEVs through decision-time planning (DTP), which is a unique method of model-based reinforcement learning (MBRL). First, the MBRL framework for the energy management problem is established, including a RL-oriented model and the DTP algorithm. The RL model consists of a deterministic nonlinear state space model and a stochastic recursive Markov Chain (MC), and the latter is constructed online and updated constantly according to new observations, which can reflect the driving condition precisely. Then, the DTP algorithm is detailed and applied. Instead of learning an overall policy for an entire driving cycle, it seeks to learn the optimal action for each encountered vehicle state, which improves the learning efficiency and realizes the real-time adaptive EMS. In the simulation, assuming that no prior information of the driving conditions is known, the proposed EMS only takes about 1–3% more fuel and 10% more battery life than dynamic programming in both off-road driving conditions and standard road cycles. The EMS significantly outperforms traditional Q-learning and rule-based strategy, verifying its optimization capability and adaptability.