A Deep Reinforcement Learning Method with Action Switching for Autonomous Navigation

Stochastic policy-based deep reinforcement learning (DRL) has successfully gained the widespread application but demands plenty of stochastic exploration to learn the environment at the initial training stage. When the agent is exposed to more complex environment, not only is the methodology inefficient, but its performance may also suffer from the issue of high variance. This paper develops a framework to accelerate the training procedure and reduce the variance by introducing a stochastic switching network, which specifically allows the agent to choose between heuristic actions and actions output by proximal policy optimization (PPO) algorithm. Instead of starting from the random actions, the agent can be effectively guided by the heuristic actions so that the navigation capability of the agent can be rapidly bootstrapped. The vanilla policy gradient (VPG) algorithm is further utilized to train the switching network, which can be jointly trained with the baseline PPO. By the experimental comparison with the baseline PPO in the customized maze environment with openAI Gym toolkit, our method greatly contributes to the more efficient execution of navigation task by means of the heuristic actions for guidance.