Hierarchical path planner combining probabilistic roadmap and deep deterministic policy gradient for unmanned ground vehicles with non-holonomic constraints

Path planning plays a vital role in autonomous driving as it needs to guide the vehicle to achieve the target position without collision while satisfying the vehicle's non-holonomic constraints. A good path planner should be able to cope with sophisticated environments and try to make the planned path as short as possible. Traditional path planning methods when being applied on four-wheel vehicles may suffer from one of the following disadvantages including conflicting with non-holonomic constraints, far from close-optimal length solution, or easy to get stuck in specific environments. In this paper, a hierarchical path planning method for a four-wheel vehicle combining probabilistic roadmap (PRM) and deep deterministic policy gradient (DDPG) is proposed to address existing problems. In the upper level, PRM is used to generate a guidance path quickly, which doesn't consider the vehicle's non-holonomic constraints but gives the sketch of the path that helps the vehicle to jump out of some dilemmas in specific environments while maintaining a relatively short path distance. In the lower level, reinforcement learning, specifically DDPG, is used to optimize the guidance path generated by PRM considering multiple factors including non-holonomic constraints and obstacle avoidance. The proposed method is validated in four different environments including a simple map, an easy-to-stuck specific map, a maze-like complex map and an office-like complex map. Results show that the proposed method could generate a smooth path satisfying non-holonomic constraints while demonstrating desirable obstacle avoidance performance. The planned path of the proposed method is superior to that of both traditional and state-of-the-art methods in terms of path length and smoothness, which could be 1.3–12.9 % shorter and 2.6–52.1 % smoother in different environments.