Safe Reinforcement Learning-Based Eco-driving Strategy for Connected Electric Vehicles at Signalized Intersection

Reinforcement learning (RL) has recently been applied to eco-driving to improve energy efficiency and reduce travel time. However, the black-box nature of RL models limits user trust in their safety performance, particularly when RL often performs poorly in scenarios beyond training environments. Safety is the most critical issue in vehicle control, making it necessary to ensure the safety of eco-driving strategies from the algorithmic level. This paper proposes a safe deep reinforcement learning (SDRL) approach to learn both safe and efficient eco-driving control strategies for connected electric vehicles (CEVs) navigating signalized intersections. The SDRL method integrates a safe-layer into conventional RL, ensuring the agent’s exploration remains safe and preventing red light violations throughout both the learning and deployment processes. Furthermore, to address the issue of delayed and sparse rewards for red light violations, a novel constraint is designed for the safe-layer. This constraint transforms the nonlinear constraints of traffic lights into time-varying linear state constraints. Experimental results highlight the effectiveness of the proposed approach in maintaining zero constraint violations for driving safety compared to conventional RL. In terms of energy saving, SDRL reduces energy consumption by an average of 11.03% compared to conventional RL and achieves 90.37% of the global optimal dynamic programming (DP) algorithm while maintaining a similar average speed.