Reinforcement Learning-Based Optimal Scheduling for Virtual Power Plant Participation in Energy and Regulation Markets

This paper addresses the day-ahead joint optimization problem of a virtual power plant (VPP) that participates in both the energy and regulation markets, and formulates a day-ahead scheduling model that explicitly captures the coupling among multiple markets. Considering that traditional analytical model-based optimization approaches suffer from high modeling complexity and limited generalization capability when facing high-dimensional decision spaces, strong nonlinear couplings, and uncertainties, a reinforcement learning-based solution is introduced to solve the proposed optimization model. Building upon the Double Deep Q-Network (DDQN) framework, a Dueling network architecture is further incorporated to decouple the state value function from the action advantage function, thereby enhancing learning efficiency and decision-making stability under diverse operating conditions.