Real-Data-Driven Offline Reinforcement Learning for Autonomous Vehicle Speed Decision Making

In recent years, reinforcement learning has demonstrated its powerful learning ability and application potential in the autonomous driving decision module, in comparison to traditional methods. However, due to the interaction with the environment, it is mostly limited to the simulation environment. Offline RL, on the other hand, can learn through fixed data sets and is considered a feasible means to push reinforcement learning to practical applications. To verify whether offline RL can exhibit excellent performance under real-world datasets, we provide a benchmark for offline RL using the Argo dataset. This paper first introduces a variety of offline RL algorithms, followed by the processing of scenes such as starting, following, and parking in the dataset, as well as the generation of the dataset and simulation environment. Different datasets were obtained through data enhancement and other methods, and several state-of-the-art(SOTA) offline RL algorithms were tested and compared with imitation learning BC. Finally, the conclusion and analysis are presented. Our code is available at https://github.com/hjcwuhuqifei/offline-rl-benchmark-by-argo.