Target-Driven Mapless Navigation for Self-Driving Car

Self-driving cars have gained a lot of research interest in both academia and industry. However, the current solutions mainly rely on either human pre-defined rules or a precise high-resolution map, which are not feasible for the unknown environments, especially when there are some extreme situations not described in the driving rules. In this paper, a new reinforcement learning based method is proposed to address these issues. First, a pre-trained VAE (Variational AutoEncoder) is used to extract representative features from road images, then PPO (Proximal Policy Optimization) algorithm is implemented to learn target-driven navigation for the self-driving car to eliminate the dependence on the map and predefined rules. Second, to improve the learning efficiency, human driving experiences are introduced and how to effectively incorporate human driving experiences into reinforcement learning is also investigated. To evaluate the performance, this algorithm is implemented and deployed in CARLA simulation environments and extensive experiments have been conducted to select the effective strategy of reusing driving experiences. The results prove that our algorithm can successfully navigate in the urban environment without a map or any predefined rules. And by integrating human driving experiences, the learning efficiency has been dramatically improved, especially when using Ratio strategy.