TY - JOUR
T1 - Joint Energy and Carbon Trading for Multi-Microgrid System Based on Multi-Agent Deep Reinforcement Learning
AU - Zhou, Yanting
AU - Ma, Zhongjing
AU - Wang, Tianyu
AU - Zhang, Jinhui
AU - Shi, Xingyu
AU - Zou, Suli
N1 - Publisher Copyright:
© 1969-2012 IEEE.
PY - 2024
Y1 - 2024
N2 - Carbon trading has emerged as an effective way to promote the renewable generation and sustainable energy development. Since carbon emissions are closely coupled to energy system, it is a challenge to design a market mechanism for joint energy and carbon trading to achieve better strategies. In this study, the energy management problem with a specific focus on joint trading in multi-microgrid system is investigated by utilizing a multi-agent deep reinforcement learning approach. Initially, a joint energy and carbon trading market is established and the dispatch optimization problem is formulated as a Markov decision process without modeling uncertainties accurately. This mechanism enables direct one-to-one energy transactions among all areas, avoiding the market clearing in traditional multi-party local energy trading markets. To enhance the learning efficiency and maintain agent privacy, an enhanced multi-agent proximal policy optimization (MAPPO) algorithm that incorporates a parameter sharing mechanism is introduced. Moreover, the recurrent neural networks (RNN) structure is leveraged to perform feature encoding for individual agents, which improves the overall feature extraction capability. Through comprehensive experiments involving various algorithms, the proposed approach reduce operating costs 14.86 % and carbon emissions 19.04 % compared with traditional MAPPO, which validates the effectiveness and performance benefits.
AB - Carbon trading has emerged as an effective way to promote the renewable generation and sustainable energy development. Since carbon emissions are closely coupled to energy system, it is a challenge to design a market mechanism for joint energy and carbon trading to achieve better strategies. In this study, the energy management problem with a specific focus on joint trading in multi-microgrid system is investigated by utilizing a multi-agent deep reinforcement learning approach. Initially, a joint energy and carbon trading market is established and the dispatch optimization problem is formulated as a Markov decision process without modeling uncertainties accurately. This mechanism enables direct one-to-one energy transactions among all areas, avoiding the market clearing in traditional multi-party local energy trading markets. To enhance the learning efficiency and maintain agent privacy, an enhanced multi-agent proximal policy optimization (MAPPO) algorithm that incorporates a parameter sharing mechanism is introduced. Moreover, the recurrent neural networks (RNN) structure is leveraged to perform feature encoding for individual agents, which improves the overall feature extraction capability. Through comprehensive experiments involving various algorithms, the proposed approach reduce operating costs 14.86 % and carbon emissions 19.04 % compared with traditional MAPPO, which validates the effectiveness and performance benefits.
KW - Carbon trading
KW - deep reinforcement learning
KW - energy management
KW - local energy trading
KW - multi-agent
UR - http://www.scopus.com/inward/record.url?scp=85189560277&partnerID=8YFLogxK
U2 - 10.1109/TPWRS.2024.3380070
DO - 10.1109/TPWRS.2024.3380070
M3 - Article
AN - SCOPUS:85189560277
SN - 0885-8950
VL - 39
SP - 7376
EP - 7388
JO - IEEE Transactions on Power Systems
JF - IEEE Transactions on Power Systems
IS - 6
ER -