TY - GEN
T1 - A Deep Reinforcement Learning Tracking Algorithm Based on Task Decomposition
AU - Yang, Kun
AU - Shen, Ao
AU - Xu, Nengwei
AU - Chen, Chen
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - The active tracking technology of unmanned aerial vehicles (UAVs) has significant applications in fields such as military operations, environmental monitoring, disaster response and traffic management. However, two major challenges greatly affect the performance of UAV active tracking in practical scenarios: (1) The presence of uncertain obstacles in the environment, which may cause issues such as occlusion and collisions, severely affects the UAV's tracking ability. (2) The randomness of target behavior can result in degraded tracking algorithm performance or even tracking failures. To address these challenges, this paper proposes a novel deep reinforcement learning algorithm based on task decomposition, which integrates the advantages of traditional heuristic methods and machine learning approaches. Firstly, a parallel neural module network is designed to decompose the UAV active tracking task into two sub-tasks: obstacle avoidance and target tracking. This task decomposition effectively reduces the complexity of the problem. Secondly, a two-stage curriculum learning framework is introduced, where the policy network of the agent is gradually trained by adjusting random obstacles to enhance training efficiency and algorithm performance. Finally, multiple simulation environments with random targets and obstacles are constructed to validate the stability and robustness of the proposed algorithm, demonstrating that it can effectively achieve tracking and obstacle avoidance in unknown environments.
AB - The active tracking technology of unmanned aerial vehicles (UAVs) has significant applications in fields such as military operations, environmental monitoring, disaster response and traffic management. However, two major challenges greatly affect the performance of UAV active tracking in practical scenarios: (1) The presence of uncertain obstacles in the environment, which may cause issues such as occlusion and collisions, severely affects the UAV's tracking ability. (2) The randomness of target behavior can result in degraded tracking algorithm performance or even tracking failures. To address these challenges, this paper proposes a novel deep reinforcement learning algorithm based on task decomposition, which integrates the advantages of traditional heuristic methods and machine learning approaches. Firstly, a parallel neural module network is designed to decompose the UAV active tracking task into two sub-tasks: obstacle avoidance and target tracking. This task decomposition effectively reduces the complexity of the problem. Secondly, a two-stage curriculum learning framework is introduced, where the policy network of the agent is gradually trained by adjusting random obstacles to enhance training efficiency and algorithm performance. Finally, multiple simulation environments with random targets and obstacles are constructed to validate the stability and robustness of the proposed algorithm, demonstrating that it can effectively achieve tracking and obstacle avoidance in unknown environments.
KW - active tracking
KW - curriculum learning
KW - deep reinforcement learning
KW - task decomposition
UR - http://www.scopus.com/inward/record.url?scp=105003271638&partnerID=8YFLogxK
U2 - 10.1109/IARCE64300.2024.00043
DO - 10.1109/IARCE64300.2024.00043
M3 - Conference contribution
AN - SCOPUS:105003271638
T3 - Proceedings - 2024 4th International Conference on Industrial Automation, Robotics and Control Engineering, IARCE 2024
SP - 184
EP - 189
BT - Proceedings - 2024 4th International Conference on Industrial Automation, Robotics and Control Engineering, IARCE 2024
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 4th International Conference on Industrial Automation, Robotics and Control Engineering, IARCE 2024
Y2 - 15 November 2024 through 17 November 2024
ER -