Enhanced Multi-Agent Proximal Policy Optimization for Multi-UAV Target Offensive-Defensive Decision

Yifan Zheng; Bin Xin; Keming Jiao; Zhixin Zhao; Yuyang Wang; Yunming Zhao

doi:10.23919/CCC58697.2023.10240070

Enhanced Multi-Agent Proximal Policy Optimization for Multi-UAV Target Offensive-Defensive Decision

Yifan Zheng, Bin Xin^*, Keming Jiao, Zhixin Zhao, Yuyang Wang, Yunming Zhao

^*Corresponding author for this work

School of Automation

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Autonomous collaborative decision-making is the key technology to achieve large-scale unmanned combat. Focus on the problem of multiple unmanned aerial vehicles' cooperative decision in target offensive and defensive combat, a multi-agent deep reinforcement learning (MADRL) based decision framework is proposed in this paper. Firstly, the simulation environment with a high-fidelity fixed-wing motion model is built. Secondly, to address the issue of high-dimension state space and credit assignment under a multi-agent environment, an enhanced multi-agent proximal policy optimization with mean-field counterfactual advantage (MAPPO - MFCOA) is proposed. Finally, the results of simulation experiments will verify the performance of the proposed approach.

Original language	English
Title of host publication	2023 42nd Chinese Control Conference, CCC 2023
Publisher	IEEE Computer Society
Pages	5986-5991
Number of pages	6
ISBN (Electronic)	9789887581543
DOIs	https://doi.org/10.23919/CCC58697.2023.10240070
Publication status	Published - 2023
Event	42nd Chinese Control Conference, CCC 2023 - Tianjin, China Duration: 24 Jul 2023 → 26 Jul 2023

Publication series

Name	Chinese Control Conference, CCC
Volume	2023-July
ISSN (Print)	1934-1768
ISSN (Electronic)	2161-2927

Conference

Conference	42nd Chinese Control Conference, CCC 2023
Country/Territory	China
City	Tianjin
Period	24/07/23 → 26/07/23

Keywords

deep reinforcement learning
multi-UAV combat
multi-agent system
proximal policy optimization

Access to Document

10.23919/CCC58697.2023.10240070

Cite this

@inproceedings{b898f62fe6e741f98a9e8c214574b3e9,

title = "Enhanced Multi-Agent Proximal Policy Optimization for Multi-UAV Target Offensive-Defensive Decision",

abstract = "Autonomous collaborative decision-making is the key technology to achieve large-scale unmanned combat. Focus on the problem of multiple unmanned aerial vehicles' cooperative decision in target offensive and defensive combat, a multi-agent deep reinforcement learning (MADRL) based decision framework is proposed in this paper. Firstly, the simulation environment with a high-fidelity fixed-wing motion model is built. Secondly, to address the issue of high-dimension state space and credit assignment under a multi-agent environment, an enhanced multi-agent proximal policy optimization with mean-field counterfactual advantage (MAPPO - MFCOA) is proposed. Finally, the results of simulation experiments will verify the performance of the proposed approach.",

keywords = "deep reinforcement learning, multi-UAV combat, multi-agent system, proximal policy optimization",

author = "Yifan Zheng and Bin Xin and Keming Jiao and Zhixin Zhao and Yuyang Wang and Yunming Zhao",

note = "Publisher Copyright: {\textcopyright} 2023 Technical Committee on Control Theory, Chinese Association of Automation.; 42nd Chinese Control Conference, CCC 2023 ; Conference date: 24-07-2023 Through 26-07-2023",

year = "2023",

doi = "10.23919/CCC58697.2023.10240070",

language = "English",

series = "Chinese Control Conference, CCC",

publisher = "IEEE Computer Society",

pages = "5986--5991",

booktitle = "2023 42nd Chinese Control Conference, CCC 2023",

address = "United States",

}

Zheng, Y, Xin, B, Jiao, K, Zhao, Z, Wang, Y & Zhao, Y 2023, Enhanced Multi-Agent Proximal Policy Optimization for Multi-UAV Target Offensive-Defensive Decision. in 2023 42nd Chinese Control Conference, CCC 2023. Chinese Control Conference, CCC, vol. 2023-July, IEEE Computer Society, pp. 5986-5991, 42nd Chinese Control Conference, CCC 2023, Tianjin, China, 24/07/23. https://doi.org/10.23919/CCC58697.2023.10240070

Enhanced Multi-Agent Proximal Policy Optimization for Multi-UAV Target Offensive-Defensive Decision. / Zheng, Yifan; Xin, Bin; Jiao, Keming et al.
2023 42nd Chinese Control Conference, CCC 2023. IEEE Computer Society, 2023. p. 5986-5991 (Chinese Control Conference, CCC; Vol. 2023-July).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Enhanced Multi-Agent Proximal Policy Optimization for Multi-UAV Target Offensive-Defensive Decision

AU - Zheng, Yifan

AU - Xin, Bin

AU - Jiao, Keming

AU - Zhao, Zhixin

AU - Wang, Yuyang

AU - Zhao, Yunming

PY - 2023

Y1 - 2023

N2 - Autonomous collaborative decision-making is the key technology to achieve large-scale unmanned combat. Focus on the problem of multiple unmanned aerial vehicles' cooperative decision in target offensive and defensive combat, a multi-agent deep reinforcement learning (MADRL) based decision framework is proposed in this paper. Firstly, the simulation environment with a high-fidelity fixed-wing motion model is built. Secondly, to address the issue of high-dimension state space and credit assignment under a multi-agent environment, an enhanced multi-agent proximal policy optimization with mean-field counterfactual advantage (MAPPO - MFCOA) is proposed. Finally, the results of simulation experiments will verify the performance of the proposed approach.

AB - Autonomous collaborative decision-making is the key technology to achieve large-scale unmanned combat. Focus on the problem of multiple unmanned aerial vehicles' cooperative decision in target offensive and defensive combat, a multi-agent deep reinforcement learning (MADRL) based decision framework is proposed in this paper. Firstly, the simulation environment with a high-fidelity fixed-wing motion model is built. Secondly, to address the issue of high-dimension state space and credit assignment under a multi-agent environment, an enhanced multi-agent proximal policy optimization with mean-field counterfactual advantage (MAPPO - MFCOA) is proposed. Finally, the results of simulation experiments will verify the performance of the proposed approach.

KW - deep reinforcement learning

KW - multi-UAV combat

KW - multi-agent system

KW - proximal policy optimization

UR - http://www.scopus.com/inward/record.url?scp=85175533585&partnerID=8YFLogxK

U2 - 10.23919/CCC58697.2023.10240070

DO - 10.23919/CCC58697.2023.10240070

M3 - Conference contribution

AN - SCOPUS:85175533585

T3 - Chinese Control Conference, CCC

SP - 5986

EP - 5991

BT - 2023 42nd Chinese Control Conference, CCC 2023

PB - IEEE Computer Society

T2 - 42nd Chinese Control Conference, CCC 2023

Y2 - 24 July 2023 through 26 July 2023

ER -

Enhanced Multi-Agent Proximal Policy Optimization for Multi-UAV Target Offensive-Defensive Decision

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this