Q-learning algorithm for path-planning to maneuver through a satellite cluster

In this paper, a path planning method for maneuvering through a satellite cluster using Q-learning is presented. An on-orbit servicing spacecraft is supposed to rendezvous with the failed central satellite of a formation and avoid collisions with the other satellites. The dynamic model of the satellite cluster is first established by Lawden equations. Then the theory of Q-learning is introduced and the reward shaping is specified to guide the learning system quickly to success. Furthermore, combining Q-learning with deep neural networks, deep Q-network (DQN) is employed when the dimension of the problem is enormous. Finally, the rendezvous mission is simulated in 2D and 3D scenarios separately to demonstrate the effectiveness of the proposed method.