Multi-agent Reinforcement Learning for Sparse Reward Tasks Using Incremental Goal Enhanced Method

Minglei Han; Zhentao Guo; Licheng Sun; Ao Ding; Tianhao Wang; Guiyu Zhao; Hongbin Ma

doi:10.1007/978-981-96-4756-9_7

Multi-agent Reinforcement Learning for Sparse Reward Tasks Using Incremental Goal Enhanced Method

Minglei Han, Zhentao Guo, Licheng Sun, Ao Ding, Tianhao Wang, Guiyu Zhao, Hongbin Ma^*

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

As the application of artificial intelligence continues to expand, complex decision-making problems such as multi-player gaming, multi-robot planning and multi-vehicle controlling have become new challenges for machine intelligence. MARL which concentrates on learning the optimal strategies of multiple agents that coexist in a shared environment, is a valid method to solve multi-agent decision-making challenges. Among MARL Algorithms, the MAPPO algorithm has won the favor of machine learning community due to its superb performance. However, the original MAPPO algorithm suffers from sparse reward issues. To overcome the sparse rewards problem and achieve sufficient learning in complex task, this paper proposes a IGE-MAPPO which uses a IGM that generates a variable-density and bi-domain reward signal, and conducts experiments on SMAC. The results show that the IGE-MAPPO algorithm can adapt to a variety of complex environment and has improved performance compared with other typical MARL algorithms.

Original language	English
Title of host publication	Computational Intelligence and Industrial Applications - 11th International Symposium, ISCIIA 2024, Proceedings
Editors	Bin Xin, Hongbin Ma, Jinhua She, Weihua Cao
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	74-85
Number of pages	12
ISBN (Print)	9789819647552
DOIs	https://doi.org/10.1007/978-981-96-4756-9_7
Publication status	Published - 2025
Externally published	Yes
Event	11th International Symposium on Computational Intelligence and Industrial Applications, ISCIIA 2024 - Beijing, China Duration: 1 Nov 2024 → 5 Nov 2024

Publication series

Name	Communications in Computer and Information Science
Volume	2466 CCIS
ISSN (Print)	1865-0929
ISSN (Electronic)	1865-0937

Conference

Conference	11th International Symposium on Computational Intelligence and Industrial Applications, ISCIIA 2024
Country/Territory	China
City	Beijing
Period	1/11/24 → 5/11/24

Keywords

Incremental Goals
Multi-Agent Proximal Policy Optimization
Sparse Rewards

Access to Document

10.1007/978-981-96-4756-9_7

Cite this

Han, M., Guo, Z., Sun, L., Ding, A., Wang, T., Zhao, G., & Ma, H. (2025). Multi-agent Reinforcement Learning for Sparse Reward Tasks Using Incremental Goal Enhanced Method. In B. Xin, H. Ma, J. She, & W. Cao (Eds.), Computational Intelligence and Industrial Applications - 11th International Symposium, ISCIIA 2024, Proceedings (pp. 74-85). (Communications in Computer and Information Science; Vol. 2466 CCIS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-981-96-4756-9_7

Han, Minglei ; Guo, Zhentao ; Sun, Licheng et al. / Multi-agent Reinforcement Learning for Sparse Reward Tasks Using Incremental Goal Enhanced Method. Computational Intelligence and Industrial Applications - 11th International Symposium, ISCIIA 2024, Proceedings. editor / Bin Xin ; Hongbin Ma ; Jinhua She ; Weihua Cao. Springer Science and Business Media Deutschland GmbH, 2025. pp. 74-85 (Communications in Computer and Information Science).

@inproceedings{ac13b9e22e2c4fc88e73d19cf466981f,

title = "Multi-agent Reinforcement Learning for Sparse Reward Tasks Using Incremental Goal Enhanced Method",

abstract = "As the application of artificial intelligence continues to expand, complex decision-making problems such as multi-player gaming, multi-robot planning and multi-vehicle controlling have become new challenges for machine intelligence. MARL which concentrates on learning the optimal strategies of multiple agents that coexist in a shared environment, is a valid method to solve multi-agent decision-making challenges. Among MARL Algorithms, the MAPPO algorithm has won the favor of machine learning community due to its superb performance. However, the original MAPPO algorithm suffers from sparse reward issues. To overcome the sparse rewards problem and achieve sufficient learning in complex task, this paper proposes a IGE-MAPPO which uses a IGM that generates a variable-density and bi-domain reward signal, and conducts experiments on SMAC. The results show that the IGE-MAPPO algorithm can adapt to a variety of complex environment and has improved performance compared with other typical MARL algorithms.",

keywords = "Incremental Goals, Multi-Agent Proximal Policy Optimization, Sparse Rewards",

author = "Minglei Han and Zhentao Guo and Licheng Sun and Ao Ding and Tianhao Wang and Guiyu Zhao and Hongbin Ma",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.; 11th International Symposium on Computational Intelligence and Industrial Applications, ISCIIA 2024 ; Conference date: 01-11-2024 Through 05-11-2024",

year = "2025",

doi = "10.1007/978-981-96-4756-9\_7",

language = "English",

isbn = "9789819647552",

series = "Communications in Computer and Information Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "74--85",

editor = "Bin Xin and Hongbin Ma and Jinhua She and Weihua Cao",

booktitle = "Computational Intelligence and Industrial Applications - 11th International Symposium, ISCIIA 2024, Proceedings",

address = "Germany",

}

Han, M, Guo, Z, Sun, L, Ding, A, Wang, T, Zhao, G & Ma, H 2025, Multi-agent Reinforcement Learning for Sparse Reward Tasks Using Incremental Goal Enhanced Method. in B Xin, H Ma, J She & W Cao (eds), Computational Intelligence and Industrial Applications - 11th International Symposium, ISCIIA 2024, Proceedings. Communications in Computer and Information Science, vol. 2466 CCIS, Springer Science and Business Media Deutschland GmbH, pp. 74-85, 11th International Symposium on Computational Intelligence and Industrial Applications, ISCIIA 2024, Beijing, China, 1/11/24. https://doi.org/10.1007/978-981-96-4756-9_7

Multi-agent Reinforcement Learning for Sparse Reward Tasks Using Incremental Goal Enhanced Method. / Han, Minglei; Guo, Zhentao; Sun, Licheng et al.
Computational Intelligence and Industrial Applications - 11th International Symposium, ISCIIA 2024, Proceedings. ed. / Bin Xin; Hongbin Ma; Jinhua She; Weihua Cao. Springer Science and Business Media Deutschland GmbH, 2025. p. 74-85 (Communications in Computer and Information Science; Vol. 2466 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Multi-agent Reinforcement Learning for Sparse Reward Tasks Using Incremental Goal Enhanced Method

AU - Han, Minglei

AU - Guo, Zhentao

AU - Sun, Licheng

AU - Ding, Ao

AU - Wang, Tianhao

AU - Zhao, Guiyu

AU - Ma, Hongbin

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2025.

PY - 2025

Y1 - 2025

N2 - As the application of artificial intelligence continues to expand, complex decision-making problems such as multi-player gaming, multi-robot planning and multi-vehicle controlling have become new challenges for machine intelligence. MARL which concentrates on learning the optimal strategies of multiple agents that coexist in a shared environment, is a valid method to solve multi-agent decision-making challenges. Among MARL Algorithms, the MAPPO algorithm has won the favor of machine learning community due to its superb performance. However, the original MAPPO algorithm suffers from sparse reward issues. To overcome the sparse rewards problem and achieve sufficient learning in complex task, this paper proposes a IGE-MAPPO which uses a IGM that generates a variable-density and bi-domain reward signal, and conducts experiments on SMAC. The results show that the IGE-MAPPO algorithm can adapt to a variety of complex environment and has improved performance compared with other typical MARL algorithms.

AB - As the application of artificial intelligence continues to expand, complex decision-making problems such as multi-player gaming, multi-robot planning and multi-vehicle controlling have become new challenges for machine intelligence. MARL which concentrates on learning the optimal strategies of multiple agents that coexist in a shared environment, is a valid method to solve multi-agent decision-making challenges. Among MARL Algorithms, the MAPPO algorithm has won the favor of machine learning community due to its superb performance. However, the original MAPPO algorithm suffers from sparse reward issues. To overcome the sparse rewards problem and achieve sufficient learning in complex task, this paper proposes a IGE-MAPPO which uses a IGM that generates a variable-density and bi-domain reward signal, and conducts experiments on SMAC. The results show that the IGE-MAPPO algorithm can adapt to a variety of complex environment and has improved performance compared with other typical MARL algorithms.

KW - Incremental Goals

KW - Multi-Agent Proximal Policy Optimization

KW - Sparse Rewards

UR - http://www.scopus.com/inward/record.url?scp=105003904660&partnerID=8YFLogxK

U2 - 10.1007/978-981-96-4756-9_7

DO - 10.1007/978-981-96-4756-9_7

M3 - Conference contribution

AN - SCOPUS:105003904660

SN - 9789819647552

T3 - Communications in Computer and Information Science

SP - 74

EP - 85

BT - Computational Intelligence and Industrial Applications - 11th International Symposium, ISCIIA 2024, Proceedings

A2 - Xin, Bin

A2 - Ma, Hongbin

A2 - She, Jinhua

A2 - Cao, Weihua

PB - Springer Science and Business Media Deutschland GmbH

T2 - 11th International Symposium on Computational Intelligence and Industrial Applications, ISCIIA 2024

Y2 - 1 November 2024 through 5 November 2024

ER -

Han M, Guo Z, Sun L, Ding A, Wang T, Zhao G et al. Multi-agent Reinforcement Learning for Sparse Reward Tasks Using Incremental Goal Enhanced Method. In Xin B, Ma H, She J, Cao W, editors, Computational Intelligence and Industrial Applications - 11th International Symposium, ISCIIA 2024, Proceedings. Springer Science and Business Media Deutschland GmbH. 2025. p. 74-85. (Communications in Computer and Information Science). doi: 10.1007/978-981-96-4756-9_7