A Deep Reinforcement Learning Tracking Algorithm Based on Task Decomposition

Kun Yang, Ao Shen, Nengwei Xu, Chen Chen*

*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

The active tracking technology of unmanned aerial vehicles (UAVs) has significant applications in fields such as military operations, environmental monitoring, disaster response and traffic management. However, two major challenges greatly affect the performance of UAV active tracking in practical scenarios: (1) The presence of uncertain obstacles in the environment, which may cause issues such as occlusion and collisions, severely affects the UAV's tracking ability. (2) The randomness of target behavior can result in degraded tracking algorithm performance or even tracking failures. To address these challenges, this paper proposes a novel deep reinforcement learning algorithm based on task decomposition, which integrates the advantages of traditional heuristic methods and machine learning approaches. Firstly, a parallel neural module network is designed to decompose the UAV active tracking task into two sub-tasks: obstacle avoidance and target tracking. This task decomposition effectively reduces the complexity of the problem. Secondly, a two-stage curriculum learning framework is introduced, where the policy network of the agent is gradually trained by adjusting random obstacles to enhance training efficiency and algorithm performance. Finally, multiple simulation environments with random targets and obstacles are constructed to validate the stability and robustness of the proposed algorithm, demonstrating that it can effectively achieve tracking and obstacle avoidance in unknown environments.

Original languageEnglish
Title of host publicationProceedings - 2024 4th International Conference on Industrial Automation, Robotics and Control Engineering, IARCE 2024
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages184-189
Number of pages6
ISBN (Electronic)9798350380323
DOIs
Publication statusPublished - 2024
Externally publishedYes
Event4th International Conference on Industrial Automation, Robotics and Control Engineering, IARCE 2024 - Chengdu, China
Duration: 15 Nov 202417 Nov 2024

Publication series

NameProceedings - 2024 4th International Conference on Industrial Automation, Robotics and Control Engineering, IARCE 2024

Conference

Conference4th International Conference on Industrial Automation, Robotics and Control Engineering, IARCE 2024
Country/TerritoryChina
CityChengdu
Period15/11/2417/11/24

Keywords

  • active tracking
  • curriculum learning
  • deep reinforcement learning
  • task decomposition

Cite this