Research on UAV Coverage Search Based on DDQN in Unknown Environments

Gaofeng Deng, Xiaolan Yao, Bo Wang, Xiao He, Qing Fei

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Citation (Scopus)

Abstract

The utilization of unmanned aerial vehicle (UAV) for area coverage search is highly sought after in both military and civil domains, including but not limited to traversal search, mission reconnaissance, patrol detection, wildfire suppression control, remote sensing mapping, agricultural preservation, and accident search and rescue. This paper focuses on the problem of area coverage search for a single UAV in environments with the presence of unknown dynamic and static targets as well as hazardous areas. Here the UAV only knows the state of a small area and remembers the actions of the last time step without long-term memory. The objective is to design an adaptive and transferable algorithm for the UAV to find all static and dynamic targets with the minimum path repetition rate in the task area where both dangerous areas and completely unknown target information exist. Because the UAV has limited ability to observe all the map information, a partially observable Markov decision process is first formulated. Then we develop a coverage search algorithm based on Double Deep Q Network (DDQN) with the help of curriculum learning. By designing multiple constraint reward functions and employing path repetition rate and target batting average as evaluation metrics, the proposed algorithm facilitates the rapid adaptation of UAVs to diverse task environments. Simulation environment and algorithm models are finally established to illustrate the efficacy of the algorithm, which shows that the proposed algorithm with curriculum learning has rapid convergence, minimal path redundancy, high target acquisition rate, robust portability, and adaptability to variations in map area, hazard zones, and target quantity.

Original languageEnglish
Title of host publicationProceedings - 2023 China Automation Congress, CAC 2023
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages2826-2831
Number of pages6
ISBN (Electronic)9798350303759
DOIs
Publication statusPublished - 2023
Event2023 China Automation Congress, CAC 2023 - Chongqing, China
Duration: 17 Nov 202319 Nov 2023

Publication series

NameProceedings - 2023 China Automation Congress, CAC 2023

Conference

Conference2023 China Automation Congress, CAC 2023
Country/TerritoryChina
CityChongqing
Period17/11/2319/11/23

Keywords

  • Double Deep Q Network
  • coverage search
  • dynamic targets
  • partially observable Markov decision process
  • unknown environment

Fingerprint

Dive into the research topics of 'Research on UAV Coverage Search Based on DDQN in Unknown Environments'. Together they form a unique fingerprint.

Cite this