TY - GEN
T1 - Cooperative Path Planning Method for Enhancing Ground-Units Survivability Based on Adaptive Q-Learning
AU - Guo, Miao
AU - Long, Teng
AU - Sun, Jingliang
AU - Li, Junzhi
N1 - Publisher Copyright:
© Beijing HIWING Scientific and Technological Information Institute 2024.
PY - 2024
Y1 - 2024
N2 - In this paper, a modified Q-Learning cooperative path planning method for enhancing the survivability of multiple ground units is proposed to alleviate the high time-consuming problem caused by wide-area road network environment and threats. The road network is established as a weighted undirected graph model based on the connection relationship of the road network nodes firstly. Then, by changing the division of the state space and the action space of the traditional Q-learning algorithm, an adaptive Q-learning algorithm based on the road network graph is proposed. The action space of the adaptive Q-learning is determined through the graph topology, and an incentive function considering threat information and target distance is designed. Further more, the adaptive Q table assessment mechanism is customized to realize the effective adaptation of complex road network scenario with single training model, whose state input is the relative start-goal distance. Finally, numerical simulations have verified the effectiveness of the proposed algorithm. Compared with sparse A* algorithm, with the increase of the scale of ground units, the planning time of the algorithm proposed is significantly reduced under the condition that the total path cost is roughly equal. Taking four and six ground units cooperative path planning for example, the average planning time of the proposed algorithm is 0.038 s and 0.041 s, which is 76.48% and 83.25% lower than the sparse A* algorithm, which verifies the efficiency and engineering practicability.
AB - In this paper, a modified Q-Learning cooperative path planning method for enhancing the survivability of multiple ground units is proposed to alleviate the high time-consuming problem caused by wide-area road network environment and threats. The road network is established as a weighted undirected graph model based on the connection relationship of the road network nodes firstly. Then, by changing the division of the state space and the action space of the traditional Q-learning algorithm, an adaptive Q-learning algorithm based on the road network graph is proposed. The action space of the adaptive Q-learning is determined through the graph topology, and an incentive function considering threat information and target distance is designed. Further more, the adaptive Q table assessment mechanism is customized to realize the effective adaptation of complex road network scenario with single training model, whose state input is the relative start-goal distance. Finally, numerical simulations have verified the effectiveness of the proposed algorithm. Compared with sparse A* algorithm, with the increase of the scale of ground units, the planning time of the algorithm proposed is significantly reduced under the condition that the total path cost is roughly equal. Taking four and six ground units cooperative path planning for example, the average planning time of the proposed algorithm is 0.038 s and 0.041 s, which is 76.48% and 83.25% lower than the sparse A* algorithm, which verifies the efficiency and engineering practicability.
KW - Adaptive Assessment Mechanism
KW - Multiple Ground Units
KW - Path Planning
KW - Q-Learning
UR - http://www.scopus.com/inward/record.url?scp=85192533935&partnerID=8YFLogxK
U2 - 10.1007/978-981-97-1083-6_52
DO - 10.1007/978-981-97-1083-6_52
M3 - Conference contribution
AN - SCOPUS:85192533935
SN - 9789819710829
T3 - Lecture Notes in Electrical Engineering
SP - 555
EP - 566
BT - Proceedings of 3rd 2023 International Conference on Autonomous Unmanned Systems (3rd ICAUS 2023) - Volume II
A2 - Qu, Yi
A2 - Gu, Mancang
A2 - Niu, Yifeng
A2 - Fu, Wenxing
PB - Springer Science and Business Media Deutschland GmbH
T2 - 3rd International Conference on Autonomous Unmanned Systems, ICAUS 2023
Y2 - 9 September 2023 through 11 September 2023
ER -