Dynamic Lag Encoding and Multi-Head Attention Improve Deep Learning Flood Forecasting

A deep learning model combining dynamic lag encoding with temporal multi-head attention significantly improves flash flood prediction accuracy across multi-hour lead times in the hilly and gully terrain of the Loess Plateau, researchers report. Developed for the Huangfuchuan Basin in the middle reaches of China’s Yellow River, the architecture addresses long-standing limitations in conventional recurrent neural networks, which typically learn rainfall-runoff relationships from static historical sliding windows. By dynamically identifying response delays between antecedent precipitation and river discharge for every individual time window, the system adapts to evolving flood conditions and prevents the loss of critical peak-flow signals.

Flooding remains one of the most severe natural hazards threatening regional sustainability and water infrastructure across China, with the middle reaches of the Yellow River presenting an acute forecasting challenge. The region is characterized by fragmented topography, steep gullies, short runoff generation pathways, and exceptionally rapid hydrological responses to localized rainstorms. Traditional basin-scale forecasting approaches, including conceptual and distributed physical hydrological models, rely on extensive meteorological parameterization and often struggle to capture the fast-moving surface runoff generated by extreme storms. While data-driven machine learning models offer nonlinear mapping capabilities with fewer explicit physical parameters, conventional deep learning approaches frequently operate as uninterpretable black boxes and struggle with the temporal lag variability inherent in rapid catchment dynamics.

To overcome these structural limitations, the research team designed the D-TMA-LSTM framework, which combines dynamic lag encoding, a temporal multi-head attention mechanism, and a long short-term memory recurrent neural network. Instead of forcing input sequences into a rigid, uniform historical window, the dynamic lag encoding module evaluates the cross-correlation between sliding-window inputs and target river discharge in real time. This process reconstructs lag-aware rainfall features that align antecedent precipitation with runoff response phases. The temporal multi-head attention module then assigns differentiated weights to these aligned time steps, enabling the model to selectively emphasize the most influential historical intervals across multiple temporal scales before feeding the transformed sequence into the long short-term memory core.

The researchers determined model input features through statistical analyses of cross-correlation and autocorrelation functions using historical rainfall, runoff, and discharge records from the Huangfuchuan Basin. The cross-correlation between rainfall and runoff peaked at a lag time of three hours, demonstrating an average statistical catchment delay, while runoff autocorrelation remained strong at a six-hour lag, confirming substantial hydrological memory. Based on these relationships, candidate lag times ranging from one to six hours were selected as model inputs. When examining dynamically selected optimal lag values across test samples, the authors observed frequent selections of one-hour and six-hour lags, alongside an even distribution across intermediate two- to five-hour intervals, proving that a single fixed lag window cannot capture the variable pacing of flash flood generation.

Across comprehensive multi-lead-time evaluation experiments spanning one-hour to six-hour forecast horizons, the integrated D-TMA-LSTM model consistently outperformed both standard long short-term memory networks and a dynamic lag variant lacking multi-head attention. At a one-hour forecast lead time, the baseline long short-term memory model exhibited a peak flow error of 29.52 percent, which decreased to 19.57 percent with the addition of dynamic lag encoding. The fully integrated D-TMA-LSTM model achieved a peak flow error of negative 7.46 percent. As the lead time extended to six hours, prediction errors increased across all configurations, but the full architecture maintained superior error control, registering a 27.94 percent peak flow error compared to 58.18 percent for the baseline network.

Statistical evaluation metrics confirmed the substantial performance advantage of the multi-head attention and lag-encoding combination across the entire hydrograph. At the one-hour lead time, the proposed model achieved a Nash-Sutcliffe efficiency of 0.959, a Kling-Gupta efficiency of 0.934, and a coefficient of determination of 0.964, alongside a root-mean-square error of 117.79 cubic meters per second. This represented a 31.74 percent reduction in root-mean-square error compared to the baseline model, which recorded 172.57 cubic meters per second. At the extended six-hour lead time, the proposed architecture sustained a Nash-Sutcliffe efficiency of 0.874 and a root-mean-square error of 206.50 cubic meters per second, whereas the baseline model declined to an efficiency of 0.811 and exhibited severe hydrograph dispersion.

Detailed examination of an individual historical event at Huangfu Hydrological Station, designated flood event 19940802, illustrated the practical hydrograph differences among the competing architectures. The baseline model exhibited marked overestimation and underestimation of discharge peaks, accompanied by a noticeable phase delay in predicting the arrival of the second major flood peak at five-hour and six-hour horizons. In contrast, the dynamic lag encoding model tracked general hydrograph trends more consistently, while the full model closely matched observed rising limbs, peak flow volumes, and recession stages. Scatter plot analyses demonstrated tight clustering along the 1:1 agreement line for low and moderate flows, with the proposed model exhibiting substantially narrower dispersion in high-discharge regimes than its counterparts.

Visualizing the internal weight allocations within the temporal multi-head attention module provided empirical insight into how the network prioritizes information during prediction. Attention weights concentrated predominantly on historical time steps immediately preceding the forecast moment, with weights tapering progressively from current time step t toward historical time step t-11. Across input feature categories, observed runoff received the highest weight, reflecting its direct representation of catchment concentration state, followed by rainfall as the primary meteorological driver. Dynamic lag features captured sustained attention, verifying that the network actively exploited explicitly aligned delay relationships to inform discharge estimates as forecast horizons lengthened.

Despite its predictive gains, the authors highlighted important methodological boundaries and dataset limitations that constrain direct operational deployment. The historical rainfall and runoff records used for model training and testing span 1979 through 1997. Over the subsequent decades, the Huangfuchuan Basin has experienced extensive land-surface changes due to vegetation restoration, soil and water conservation projects, and engineering interventions, all of which alter hydrological runoff generation and concentration timing. Furthermore, high-discharge peak observations were relatively scarce in the training data, leading to underestimation during rare extreme flows. Future work must evaluate recent independent flood records under modern catchment conditions and incorporate numerical weather predictions to extend reliable lead times.

Overall, the study demonstrates that introducing explicit hydrological structure into deep learning pipelines resolves key shortcomings of standard black-box recurrent networks. By replacing rigid temporal input windows with dynamically calculated delay parameters and interpretable multi-head attention weighting, the architecture preserves steep flood-peak morphology and mitigates forecast degradation across multi-hour lead times. These findings establish a viable structural pathway for improving data-driven early warning systems in steep, flood-prone river basins where rapid concentration times severely compress emergency response windows.

The performance improvements observed in the Huangfuchuan Basin align with broader advances in structure-enhanced hydrological forecasting across Chinese river catchments. Li and colleagues tested an RS-LSTM-Transformer framework for multi-lead-time forecasting, recording validation Nash-Sutcliffe efficiency values of 0.953 at a one-hour horizon and 0.875 at six hours. Similarly, Luo and colleagues found that an SHG-LSTM architecture reduced root-mean-square error by 5.0 percent, mean absolute error by 6.7 percent, and flood-volume error by 6.5 percent relative to conventional networks while curbing multi-step underestimation. In the Bahe Basin, Wang and colleagues showed that integrating temporal attention into baseline encoder-decoder models raised six-hour forecast efficiency from 0.89 to 0.920 and reduced the interquartile range of relative peak-flow error by roughly 30.7 percent. These parallel outcomes confirm that explicitly structuring historical dependencies helps neural architectures retain critical hydrological signals.

Tracking performance degradation across intermediate forecast steps further illuminates why the combined framework preserves stability under extended lead times. As the prediction horizon expanded from one to six hours, Nash-Sutcliffe efficiency decreased by 0.101 in the baseline long short-term memory network and by 0.110 in the dynamic lag model, but fell by only 0.085 in the D-TMA-LSTM system. The coefficient of determination exhibited an identical trend, declining by 0.103 and 0.108 in the simpler configurations compared with a 0.082 drop in the full model. At the six-hour horizon, the dynamic lag model without attention reached a Nash-Sutcliffe efficiency of 0.822, a Kling-Gupta efficiency of 0.814, and a determination coefficient of 0.825. Incorporating multi-head attention elevated those respective indicators to 0.874, 0.825, and 0.882, demonstrating that simultaneous focus across diverse timescale dependencies mitigates the cumulative routing uncertainties that typically erode extended runoff predictions.

The authors also noted that the lag values identified by dynamic lag encoding represent effective mathematical response lags within sliding windows rather than direct physical concentration times. Because six hours formed the upper boundary of the search range, its frequent selection may reflect both genuine delayed runoff processes and an edge effect of the parameter window, meaning maximum search bounds must be calibrated individually for different river basins. In addition, runoff mechanisms differ substantially between flash floods triggered by intense, short-duration cloudbursts and prolonged events governed by basin concentration dynamics. To refine future operational forecasting, the researchers suggested incorporating pre-peak rainfall intensity, storm duration, cumulative precipitation, and data assimilation schemes, while using bootstrap techniques to evaluate sensitivity to event partitioning.

Subject of Research: Deep learning flood forecasting using dynamic lag encoding and temporal multi-head attention

Article Title: A deep learning model for flood forecasting based on dynamic lag encoding and multi-head attention

Article References: Cui, B., Zhu, D., Ma, J., Jiao, X., Bian, X., Fang, C., Hao, X., & Zhao, Y. (2026). A deep learning model for flood forecasting based on dynamic lag encoding and multi-head attention. iScience, 29(11), Article 118177. https://doi.org/10.1016/j.isci.2026.118177

Image Credits: AI Generated

DOI: 10.1016/j.isci.2026.118177

Keywords: flood forecasting, deep learning, dynamic lag encoding, temporal multi-head attention, LSTM, Huangfuchuan Basin, Yellow River, Loess Plateau, flash floods, rainfall-runoff modeling, hydrological forecasting, peak flow prediction, Nash-Sutcliffe efficiency, lead-time forecasting, data-driven hydrology, attention mechanism