Cooperative Path Planning Method for Enhancing Ground-Units Survivability Based on Adaptive Q-Learning

In this paper, a modified Q-Learning cooperative path planning method for enhancing the survivability of multiple ground units is proposed to alleviate the high time-consuming problem caused by wide-area road network environment and threats. The road network is established as a weighted undirected graph model based on the connection relationship of the road network nodes firstly. Then, by changing the division of the state space and the action space of the traditional Q-learning algorithm, an adaptive Q-learning algorithm based on the road network graph is proposed. The action space of the adaptive Q-learning is determined through the graph topology, and an incentive function considering threat information and target distance is designed. Further more, the adaptive Q table assessment mechanism is customized to realize the effective adaptation of complex road network scenario with single training model, whose state input is the relative start-goal distance. Finally, numerical simulations have verified the effectiveness of the proposed algorithm. Compared with sparse A* algorithm, with the increase of the scale of ground units, the planning time of the algorithm proposed is significantly reduced under the condition that the total path cost is roughly equal. Taking four and six ground units cooperative path planning for example, the average planning time of the proposed algorithm is 0.038 s and 0.041 s, which is 76.48% and 83.25% lower than the sparse A* algorithm, which verifies the efficiency and engineering practicability.