The research, led by Zexin Mu and colleagues at the State Grid Heilongjiang Electric Power Company Limited Electric Power Research Institute, together with a collaborator at Tianjin Pingyun Electric Power Technology, tackles a stubborn problem in modern grid operations. As China pushes toward its dual carbon goals and builds power systems dominated by renewable energy, the volatility and intermittency of wind and solar output have become the grid’s greatest vulnerability. Under extreme weather, the spatial and temporal distribution of renewable generation is tightly constrained by meteorological conditions, and the risk of a sudden supply-demand imbalance grows sharply. Traditional defense strategies, which span pre-disaster prevention, during-disaster emergency response, and post-disaster recovery, have helped, but they lean on precise physical models and deterministic optimization solvers that struggle when system states change rapidly and control variables multiply.

The core difficulty, the authors explain, is twofold. First, physics-based methods suffer from accumulated modeling errors when extreme weather makes it nearly impossible to obtain accurate parameters in real time. Second, with large-scale renewable integration, the dimensionality of the control problem explodes, imposing computational burdens that make it hard to generate feasible pre-control schemes within the limited decision-making windows available to operators. Deep reinforcement learning offers an alternative: by letting an agent learn through trial-and-error interaction with a simulated environment, it can discover effective policies without requiring perfect models. But existing DRL approaches have their own flaws, including blind early exploration, a tendency to generate physically impossible actions, and a narrow focus on single resource types rather than the coordinated use of multiple flexible assets.

To close those gaps, the team built their method on an improved version of the deep deterministic policy gradient, or DDPG, algorithm, fusing physical knowledge with data-driven learning. The centerpiece is a reward function embedded with the physical rules of active regulation flexibility. Rather than treating equipment constraints as an afterthought—simply clipping actions or adding generic penalties—the researchers introduced equipment operating boundary constraint penalties and regulation margin incentives directly into the policy update process. The agent learns to proactively perceive and maintain the system’s flexibility margin throughout training. Crucially, the constraint-penalty pairs are constructed dynamically according to the severity of the extreme weather event, so the penalty intensity adapts as environmental conditions shift, unlike the static action clipping used in methods such as TD3.

The framework also incorporates two additional mechanisms borrowed and refined from recent reinforcement learning research. A dynamic entropy regularization term is added to the action-value function, adaptively balancing exploration against exploitation. The entropy term rewards policies whose action distributions remain relatively uniform, encouraging the agent to try low-probability actions during early training and avoid premature convergence to suboptimal strategies. Meanwhile, a twin-delayed network structure employs two independent Critic networks that estimate Q-values separately, with the smaller of the two estimates taken as the target, while the Actor network’s update frequency is delayed. This combination mitigates the notorious Q-value overestimation problem that plagues standard DDPG, improving the robustness of value estimation and the stability of policy training. The authors note that, unlike the original TD3 algorithm, their approach integrates the twin-delayed structure uniformly into the knowledge-data fusion framework, forming a synergistic optimization with dynamic constraint construction and physical rule embedding.

On the modeling side, the researchers constructed mechanism models for wind power and photovoltaic output, including a piecewise wind power curve governed by cut-in, rated, and cut-out wind speeds, and a photovoltaic model that accounts for real-time irradiance and temperature effects. An hourly time-series energy storage model tracks the state of charge of storage units, with charging and discharge efficiencies set at 0.95 and initial state of charge fixed at 50 percent of rated capacity each day. These models, combined with 24-hour time-series meteorological data, allow the team to quantify the supply-demand imbalance risk under extreme weather scenarios before the AI ever begins generating control schemes.

The experiments drew on publicly available datasets, including the CNPS-Met meteorological dataset built from ERA5 reanalysis data and observations from 2,598 national ground stations, spanning 31 years of daily gridded data with 11 categories of high-impact weather events. Load data came from the China Southern Power Grid load forecasting competition dataset within the EnergyTS collection, while energy storage and unit parameters referenced clean energy scenario data from the National Basic Science Data Center. Extreme weather events were selected using meteorological thresholds: typhoon scenarios required sustained winds above 17.2 meters per second—Beaufort scale 8 or above—lasting more than six hours, while ice disaster scenarios required daily average temperatures below minus 5 degrees Celsius accompanied by precipitation. The data were split into training, validation, and test sets at a ratio of 7:1.5:1.5, and all experiments ran on an Intel Xeon Gold 6248R CPU with an NVIDIA RTX 4090 GPU and 128 GB of memory.

The results are striking. Under extreme weather scenarios, the proposed method achieved a total pre-control cost of 120.6 CNY, a success rate of 94.7 percent, and a violation rate of 0.0 percent, outperforming five benchmark schemes including preventive control of successive failures, a multi-level operation method, a risk-averse deep learning approach, mixed-integer linear programming, and standard DDPG. The total cost was 13.1 percent lower than the second-best method, and the success rate exceeded MILP by 3.1 percentage points and standard DDPG by 15.4 points. On computational efficiency, the method’s average decision time of 675.9 milliseconds was slower than the risk-averse deep learning method’s 198.3 milliseconds but vastly faster than MILP’s 47.8 seconds, comfortably meeting the real-time requirements of day-ahead scheduling. Ablation experiments confirmed that each improved module contributes synergistically: removing the twin-delayed network raised total cost by 28.8 percent and cut the success rate by 11.8 percentage points, while removing boundary clipping or the penalty term allowed violation rates to climb to 6.2 and 4.8 percent respectively.

Visualization experiments under a compound extreme weather event—strong wind accompanied by low temperature lasting eight hours—illustrated the difference in physical realism. The proposed method kept the energy storage state of charge above 0.33 during the extreme period, never violating the lower bound of 0.2, whereas standard DDPG let it drop to 0.12 between 12:00 and 14:00. Storage power under the proposed scheme stayed smoothly within plus or minus 10 MW, well below the 15 MW rating, while standard DDPG pushed discharge to 18 MW beyond the rated boundary. The proposed method also raised conventional unit output smoothly from 24 MW to roughly 30 MW to provide orderly power support, while standard DDPG exhibited erratic fluctuations. In scenario identification tests across typhoon, ice disaster, extreme heat, and compound events, the method achieved 94.0 percent overall accuracy compared with 74.5 percent for standard DDPG, and generalization experiments on modified IEEE 33-bus and 123-bus systems showed success rates of 93.2 and 91.8 percent with zero violations. Together, these findings suggest that embedding physical knowledge deep inside reinforcement learning could give grid operators a powerful new tool for staying ahead of the weather’s worst moods.

Subject of Research: Deep reinforcement learning for pre-control of power imbalance risk under extreme weather

Article Title: Pre-control scheme generation for extreme-weather power imbalance risk using multi-resource frequency regulation and deep reinforcement learning

Article References: Mu, Z., Hu, Y., Chen, H., Zheng, J., & Wang, Z. (2026). Pre-control scheme generation for extreme-weather power imbalance risk using multi-resource frequency regulation and deep reinforcement learning. Discover Artificial Intelligence, 6(1), Article 1379. https://doi.org/10.1007/s44163-026-02335-4

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02335-4

Keywords: deep reinforcement learning, power grid resilience, extreme weather, DDPG, energy storage, frequency regulation, renewable energy, power imbalance risk, entropy regularization, twin-delayed networks, pre-control schemes, smart grid