Al-Generated Expert Data Assisted Reinforcement Learning for Multi-agent Cooperative Tasks

We can use multi-agent reinforcement learning algorithms such as Multi-Agent Deep Deterministic Policy Gradient (MADDPG) to effectively solve the organization problem of multi-agent systems. However, the training process of these algorithms is challenging due to sparse rewards, which means it typically takes a long time to finish the training. To address the challenge, we propose the Large Language Model (LLM) Assisted MADDPG (LLM-MADDPG) algorithm that utilizes LLM to guide the training of multi-agent systems. The core original contribution is two-fold. First is the dual-track mechanism, which combines the autonomous policy learning capability of the distributed Actor network with the cross-domain, high-level priori knowledge guidance provided by the Large Language Model (LLM). They both work together to improve training efficiency. The second is knowledge quality assurance through human-in-the-loop screening. The expert strategy knowledge generated by LLM is refined through manual screening and transformed into a code-based reward function to ensure logical rationality and accelerate the generation of correct guidance from LLM.