Machine Learning Breakthrough Doubles Accuracy of Two-to-Six-Week Weather Forecasts

Weather forecasts have long been reliable only within a narrow window. Beyond roughly two weeks, the atmosphere’s inherent chaos overwhelms even the most sophisticated simulation models, leaving forecasters with little more than statistical averages to offer. Now, a team of researchers led in part by scientists at the Massachusetts Institute of Technology has developed a machine-learning technique that dramatically extends the useful range of forecasting, doubling predictive accuracy in the critical two-to-six-week timeframe and, for the first time, beating simple climatological averages in that range. The work, described in a new paper in the journal Nature, placed first in a prominent international forecasting competition and could reshape how energy operators, farmers, and emergency managers prepare for severe weather around the world.

The stakes of extending forecast range are not abstract. In February 2021, three severe winter storms struck Texas, triggering the worst energy infrastructure failure in the state’s history. At least 246 people died as a result of the cascading disruptions, which cut off food, water, heat, and electricity across wide swaths of the state. According to the researchers, if energy officials had received reliable warnings weeks in advance, they could have taken concrete mitigating steps, such as arranging additional trucks of natural gas to serve disrupted areas before the crisis unfolded. The Texas disaster is a vivid illustration of why the subseasonal window, spanning roughly two to six weeks ahead, is considered a key planning horizon for emergency response, agriculture, and the energy sector.

To understand why forecasts decay so quickly, the researchers point to a foundational insight from MIT’s own history. In the 1970s, MIT Professor Edward Lorenz developed chaos theory, describing how infinitesimally small differences in the atmosphere’s initial conditions compound into completely different weather outcomes. Lorenz proposed a theoretical two-week limit beyond which deterministic weather prediction becomes impossible, because the noise in the system grows until it overwhelms any initial signal. That limit, says Judah Cohen, an MIT research scientist and co-author of the new study, exerted a strong influence on the field and may have discouraged researchers from even attempting forecasts beyond two weeks. Cohen, however, never fully accepted the barrier. While the atmosphere can in principle produce an infinite number of patterns, he argues, it tends to fall into favorite patterns repeatedly, so that once one pattern emerges, the sequence of subsequent dominoes becomes, to some degree, predictable.

Today’s best forecasts come from two main families of models. Dynamical models numerically simulate the physical evolution of the atmosphere and oceans over time, while artificial intelligence models learn to make predictions from vast archives of historical weather data. AI models have recently shown impressive gains at short ranges, but at longer time horizons they perform even worse than dynamical models. The gap between these regimes is what forecasters call the subseasonal range, and Cohen describes it bluntly as the predictability desert. In the short range, forecasters can exploit the initial conditions of the climate system and atmosphere; on seasonal and longer scales, slow-varying boundary forces such as El Niño provide leverage. The subseasonal range, caught in between, can take advantage of neither, and in most cases, Cohen says, forecasting there has been no different from rolling dice at a casino.

The breakthrough came from an idea credited to Cohen’s longtime collaborator Lester Mackey, a researcher at Microsoft: rather than trying to build a better forecasting model from scratch, use machine learning to quantify and remove the errors of existing ones. The team built what they describe as a wrapper, or postprocessing layer, that sits on top of an existing forecasting system. Unlike the more simplistic error-correction schemes used by other forecasting systems, the new layer uses AI to learn from a forecasting system’s historical mistakes and shifts its forecasts accordingly. Because the target range demands probabilistic forecasts, in which the likelihood of a range of outcomes is given rather than a single deterministic prediction, the correction operates on distributions of possibilities. Crucially, the model is adaptive: as more forecasts and observational data become available, it continuously refines its correction.

The researchers trained and developed the model using MIT Engaging, a supercomputing cluster hosted by MIT’s Office of Research Computing and Data, but the finished model does not require a supercomputer to run, a practical detail that lowers the barrier for meteorological services worldwide. To validate the approach, the team last year submitted three models incorporating their postprocessing layer to the European Center for Medium-Range Weather Forecasts, widely regarded as the best forecasting center in the world. ECMWF holds forecasting competitions between models each season, providing a rigorous, independent benchmark. The winning submission combined ECMWF’s dynamical forecasting model with another model and then applied the postprocessing correction on top. The result was a doubling of the original model’s predictive power in the subseasonal range and, for the first time, a demonstrated improvement over simply relying on long-term climatological averages.

That last achievement may be the most consequential. Until now, Cohen notes, no one had shown a model that predicts weather better than long-term averages for subseasonal forecasts; previous systems had essentially all been worse, to the point that flipping a coin could outperform them, and researchers had questioned whether investing in models for this range was worthwhile at all. The team’s models consistently beat climatology. Equally important, the approach produced a statistically significant improvement in predicting extreme weather events, such as flooding and cold spells. This is a notoriously difficult target. When ECMWF developed its own non-AI error correction, the corrected model actually got worse at predicting extremes. Cohen attributes that failure to the nature of traditional bias correction, which always pulls forecasts toward the mean, whereas extreme weather sits on the fringe of the probability distribution. The AI-driven correction, by contrast, learned to preserve and sharpen signals at the edges.

The technique’s versatility is another of its strengths. In the Nature paper, the team showed that the method could improve land forecasts across most regions of the globe. They also applied the bias-correction wrapper to other forecasting models and found it improved those as well, lifting even lesser models to parity with ECMWF’s forecasts. Cohen emphasizes that the model is an add-on wrapper rather than a replacement: the team used it with ECMWF’s dynamical model because that model was considered the best, but it can be applied to any model, and it even improves models that have already been bias-corrected using another AI algorithm. The improvement is robust, and the tool is open source and low cost, making it accessible to national weather services that lack the computational resources of the major forecasting centers.

Real-world adoption may already be beginning. Cohen says the meteorological office for the Republic of Kenya has reached out about implementing the model to improve forecasting of the annual monsoon season, with the goal of boosting agricultural yields. The potential applications extend across sectors: farmers could time fertilization around expected rain, protect crops from incoming freezes during winter harvests, and take steps to avoid drought losses. Energy and emergency management officials stand to benefit as well. Cohen recounts conversations with energy officials who explain that two days of warning before an ice storm is not enough time to prepare, but two weeks could be sufficient to make adequate arrangements, from securing fuel supplies to staging repair crews. The work was supported in part by the U.S. National Science Foundation, the Carlos Chagas Filho Foundation for Research Support, and the European Union, and the researchers hope that other groups around the world will build on the openly available model to push the predictability desert further back.

Subject of Research: Machine learning postprocessing to improve subseasonal weather forecasting accuracy from two to six weeks ahead

Article Title: Researchers improve weather forecasts looking two to six weeks out

Article References: Researchers improve weather forecasts looking two to six weeks out. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: subseasonal forecasting, machine learning, weather prediction, MIT, ECMWF, error correction, extreme weather, chaos theory, Nature, energy planning, emergency preparedness, climatology