Extreme temperature events are no longer isolated anomalies appearing once in a generation. As the climate warms, heat domes, sudden freezes and other abrupt weather disturbances are occurring more frequently, lasting longer and producing greater impacts on energy systems, transportation networks and public safety. Yet forecasting these events remains unusually difficult. Their rapid onset, complex spatial structure and departure from historical patterns can expose the weaknesses of conventional time-series models, which often assume that the statistical behavior of a system remains relatively stable over time.
Researchers at the Hangzhou Institute for Advanced Study of the University of Chinese Academy of Sciences have developed a deep-learning architecture designed to address this problem. Called Hankelformer, the model combines structured time-series augmentation with contrastive learning to improve the prediction of non-stationary and extreme events. According to the team’s findings, published in National Science Review, Hankelformer consistently outperformed leading forecasting methods across a wide range of benchmark datasets, achieving improvements in mean squared error of up to 34 percent.
The central challenge in extreme-event forecasting is that the most important patterns may be rare, short-lived and highly nonlinear. A model trained primarily on ordinary conditions can learn to predict the average behavior of a system while failing precisely when conditions become dangerous. A sudden collapse in temperature, an explosive rise in heat or a rapid shift in a multivariate weather field may not resemble the examples contained in the training data. Hankelformer was designed to make these hidden transitions easier to identify by transforming sequential information into additional, structurally meaningful views without destroying the order of events.
Its first major component is a structured augmentation module based on Hankel matrices. In time-series analysis, a Hankel matrix is formed by arranging overlapping segments of a sequence along successive rows or columns. For a signal represented as a sequence of observations, the matrix therefore contains delayed versions of the same signal, allowing the model to examine relationships among multiple time lags at once. Instead of treating each measurement as an isolated point, this construction exposes the local temporal geometry of the data and can reveal recurring transitions, changing oscillations and short-term dependencies that may be difficult to detect in the original representation.
The researchers use this matrix-based transformation to generate delay-embedding-inspired versions of the input sequence. Delay embedding is a technique associated with dynamical-systems analysis in which a system’s underlying state is reconstructed from observations collected at different time delays. The resulting views are intended to preserve the essential structure of the original trajectory while presenting it in a form that emphasizes local spatiotemporal relationships. Hankelformer’s augmentation is designed to be topologically equivalent to the original sequence, meaning that it changes the representation rather than arbitrarily altering the system’s temporal identity. This distinction is critical: random perturbations can make training data more diverse, but they may also introduce patterns that are physically meaningless or violate the sequence’s chronology.
The second innovation is a dual-stream contrastive learning framework. Hankelformer processes both the original sequence and its Hankel-augmented counterpart through Transformer encoders with shared weights. The two streams therefore use the same feature-extraction mechanism, but they receive different views of the underlying data. During training, a contrastive objective encourages the model to produce similar representations for the original and augmented versions of the same sequence. At the same time, representations associated with unrelated examples are pushed apart. This process teaches the network to focus on features that remain stable across valid transformations instead of relying on superficial details tied to a single input format.
This alignment strategy is particularly important for non-stationary data. When a system shifts from normal behavior to an extreme regime, the raw distribution of observations can change dramatically. A model that has memorized the precise appearance of historical sequences may then become unreliable. By requiring the original and augmented views to agree at the representation level, contrastive learning encourages Hankelformer to capture more invariant characteristics of the dynamics. The approach also helps the Transformer distinguish meaningful changes in the system from distortions caused by noise, altered sampling patterns or distribution shifts between training and deployment.
The team evaluated the architecture on nine benchmark datasets covering energy, transportation and extreme-weather forecasting. Three datasets were created to represent major real-world events: the winter storm that struck Texas in 2021, the Pacific Northwest heat dome of the same year and an extreme heat event recorded in the Antarctic Peninsula in 2020. These cases are particularly demanding because they combine high-dimensional measurements with unusual and rapidly evolving conditions. Across the experiments, Hankelformer achieved state-of-the-art results, with the largest reported advantage reaching a 34 percent reduction in mean squared error compared with leading baseline models. The results indicate that the model’s benefits are not limited to one type of climate event or one forecasting domain.
The researchers also tested Hankelformer on a 90-dimensional chaotic Lorenz system, a synthetic environment widely used to examine forecasting under nonlinear and sensitive dynamics. Because chaotic systems amplify small errors, they provide a stringent test of a model’s ability to identify useful structure in noisy signals. Hankelformer maintained relatively low prediction error even when strong Gaussian noise was added to the observations. This performance suggests that the model is not simply fitting clean, short-term correlations, but is learning a more robust representation of the system’s evolving state.
Ablation experiments clarified why the architecture works. When the Hankel augmentation was used without the contrastive loss, performance deteriorated rather than improving. The finding highlights a potential danger of adding structured transformations to complex data: multiple representations may contain complementary information, but they can also create incompatible feature spaces that make optimization unstable. Contrastive learning appears to provide the mechanism that aligns these views, allowing the network to benefit from the extra temporal structure without becoming confused by it. In this sense, Hankelformer’s strength comes not from either component alone, but from the interaction between structured augmentation and representation-level agreement.
The implications extend beyond weather prediction. More dependable forecasts of extreme temperatures could help grid operators anticipate sudden changes in electricity demand, support emergency planning and improve the scheduling of renewable-energy resources. Similar methods could be applied to traffic systems, where congestion can emerge abruptly, or to industrial and financial monitoring, where rare deviations may carry outsized consequences. The architecture may also be relevant to other safety-critical applications in which historical averages provide little protection against rapidly developing failures.
Hankelformer does not eliminate the fundamental uncertainty associated with chaotic and climate-driven systems, and computational forecasting models remain dependent on the quality, coverage and resolution of their input data. Nevertheless, its results point toward a broader strategy for machine learning on real-world time series: rather than relying only on larger models or more historical observations, researchers can create mathematically structured views of existing data and train networks to identify the information shared across those views. By combining the geometric perspective of delay embeddings with the flexibility of Transformers and the stability of contrastive learning, the new architecture offers a promising route toward forecasting the extreme events that conventional models most often miss.
Subject of Research: Hankelformer, a deep-learning architecture for forecasting non-stationary time series and extreme weather events
Web References: https://doi.org/10.1093/nsr/nwag456
References: National Science Review, DOI: 10.1093/nsr/nwag456
Keywords: Hankelformer, extreme weather forecasting, climate change, deep learning, Transformer, Hankel matrix, contrastive learning, time-series forecasting, delay embedding, non-stationary systems, chaotic dynamics, noise robustness
