TY - JOUR
T1 - Active sensitivity coefficient-guided reinforcement learning for power grid real-time dispatching
AU - Yang, Nan
AU - Song, Xuri
AU - Huang, Yupeng
AU - Chen, Ling
AU - Yu, Yijun
N1 - Publisher Copyright:
© 2024 Elsevier B.V.
PY - 2025/3
Y1 - 2025/3
N2 - The integration of renewable energy sources into power systems and the expansion of grid scale have introduced a higher degree of operational unpredictability, thereby increasing the risk of safety hazards such as transmission line overloads. Existing reinforcement learning algorithms that incorporate expert knowledge aim to facilitate secure dispatching of the power grid. However, these algorithms, which adopt strategies akin to imitation learning during the learning process, do not effectively address systemic constraints such as line limits and power balance. In response to this issue, this study introduces a novel active sensitivity coefficient-guided reinforcement learning method for real-time power grid dispatch (SRL-Dispatching). Firstly, a sensitivity-based reinforcement learning framework is proposed using the Soft Actor-Critic (SAC) algorithm. Secondly, a sensitivity coefficient-based method for compressing the action space boundary is proposed to address the security implications associated with transmission line overloads. Subsequently, a feasible domain projection approach is introduced to ensure that dispatch operations adhere to safety constraints. By employing sensitivity factors to guide the learning process of the agent in managing line overloads and ensuring safe operation, the precision and robustness of dispatch strategies in the power system environment are significantly enhanced. Simulation results using the SG-126 power grid simulator indicate that SRL-Dispatching accelerates training by a factor of 9.8 compared to state-of-the-art RL methods, with comparable decision-making times. Across various load levels, SRL-Dispatching demonstrates superior performance in terms of renewable energy integration, line overload management, and power balance control.
AB - The integration of renewable energy sources into power systems and the expansion of grid scale have introduced a higher degree of operational unpredictability, thereby increasing the risk of safety hazards such as transmission line overloads. Existing reinforcement learning algorithms that incorporate expert knowledge aim to facilitate secure dispatching of the power grid. However, these algorithms, which adopt strategies akin to imitation learning during the learning process, do not effectively address systemic constraints such as line limits and power balance. In response to this issue, this study introduces a novel active sensitivity coefficient-guided reinforcement learning method for real-time power grid dispatch (SRL-Dispatching). Firstly, a sensitivity-based reinforcement learning framework is proposed using the Soft Actor-Critic (SAC) algorithm. Secondly, a sensitivity coefficient-based method for compressing the action space boundary is proposed to address the security implications associated with transmission line overloads. Subsequently, a feasible domain projection approach is introduced to ensure that dispatch operations adhere to safety constraints. By employing sensitivity factors to guide the learning process of the agent in managing line overloads and ensuring safe operation, the precision and robustness of dispatch strategies in the power system environment are significantly enhanced. Simulation results using the SG-126 power grid simulator indicate that SRL-Dispatching accelerates training by a factor of 9.8 compared to state-of-the-art RL methods, with comparable decision-making times. Across various load levels, SRL-Dispatching demonstrates superior performance in terms of renewable energy integration, line overload management, and power balance control.
KW - Active sensitivity coefficient
KW - Real-time dispatching optimization
KW - Renewable energy integration
KW - Sensitivity reinforcement learning
KW - Soft actor and critic
UR - http://www.scopus.com/inward/record.url?scp=85210126192&partnerID=8YFLogxK
U2 - 10.1016/j.epsr.2024.111267
DO - 10.1016/j.epsr.2024.111267
M3 - Article
AN - SCOPUS:85210126192
SN - 0378-7796
VL - 240
JO - Electric Power Systems Research
JF - Electric Power Systems Research
M1 - 111267
ER -