Continuous control for moving object tracking of unmanned skid-steered vehicle based on reinforcement learning

Skid Steering vehicles are being widely used due to their robust mechanical structure and high maneuverability. Moving object tracking for unmanned skid-steered vehicle (USSV) is a challenging task that requires delicate actions to ensure a smooth trajectory and accurate response between ego vehicle and the moving object. However, inevitable slipping and sliding of the tire that makes the vehicle difficult to control and accurate model of USSV are hard to describe. This paper proposes a real-time moving object tracking system with continuous actions for USSV base on a reinforcement learning algorithm named Twin Delay Deterministic Policy Gradient (TD3). The capacity of the replay buffer, which is critical in the training process, changes softly as the training episodes increases. We added two control group models with a fixed capacity of replay buffer and trained the RL agent from scratch in the gazebo environment. By observing the training and validation results, we can conclude that our RL model performs well for moving target tracking, and the model with soft updated replay buffer has high efficiency in the training process and high accuracy in the evaluation process.