Multiagent reinforcement learning with evolution for multitarget tracking by unmanned aerial vehicle swarm

Multitarget tracking has immense potential in both military and civilian applications. For an unmanned aerial vehicle (UAV) swarm, a critical challenge in multitarget tracking is how to coordinate multiple unmanned aerial vehicles to continuously and accurately track multiple targets. This paper considers the cooperative decision-making problem for multitarget tracking by multiple unmanned aerial vehicles with limited sensing range in a dynamic environment. Meanwhile, a multiagent advantage actor critic with evolution, named MAA2CE, is proposed to learn the cooperative tracking policies for a UAV swarm. During training, each UAV is viewed as an agent with a policy network, making decisions based on its own information and policy. The prioritized experience replay is adopted to take full advantage of the valuable experience for learning. Considering the tracking performance of agents, the high performance agent can replicate its own network parameters to the other agents with a certain probability by network evolution. The experimental results demonstrate that the proposed algorithm is superior to three peer algorithms in learning efficiency, can acquire better collaborative tracking policies, and significantly improves the collaborative multitarget tracking proficiency of the UAV swarm.