How Good Is a Good Hydrological Model? New Benchmarks Redefine the Yardstick for River Flow Simulations

For decades, hydrologists have graded their river flow models against a single, unforgiving scale. A model’s skill at reproducing observed streamflow is typically condensed into a single number, most famously the Nash-Sutcliffe efficiency, and that number has often been judged against fixed thresholds published in widely cited guidelines. A value above 0.7 might be declared good, anything below deemed questionable, regardless of whether the model is simulating a rain-drenched mountain catchment in Switzerland or a parched, evaporation-dominated basin in central Spain. A new study published in Hydrology and Earth System Sciences argues that this long-standing practice is fundamentally misleading, and it offers a concrete, data-driven alternative that could reshape how thousands of modelling studies are evaluated.

Jan Seibert, Marc Vis, and Sandra Pool, based at the University of Zurich and the Swiss Federal Institute of Aquatic Science and Technology, set out to answer a deceptively simple question: what level of model performance should we actually expect in a given catchment? Their answer takes the form of two reference points, a lower benchmark and an upper benchmark, computed for more than 6000 catchments across thirteen large-sample datasets spanning Australia, Brazil, Central Europe, Chile, Denmark, France, Germany, Great Britain, Luxembourg, Spain, Sweden, Switzerland, and the United States. The lower benchmark captures what an entirely uninformed model, one that has never seen a single drop of local streamflow data, can achieve. The upper benchmark represents what a well-calibrated model can realistically deliver given the local data and model structure. Together, they frame a window of achievable performance that varies dramatically from place to place.

The case against fixed thresholds rests on a striking empirical observation. In previous work, the team showed that a completely uncalibrated bucket-type model, run with essentially arbitrary parameter values, dropped below a Nash-Sutcliffe efficiency of zero in only one out of ten catchments and reached values of 0.8 or higher in humid or snow-dominated regions. To put that in perspective, a value of zero corresponds merely to predicting a constant discharge equal to the annual mean, a benchmark that is trivially easy to beat. Yet an uncalibrated model, knowing nothing about the local river, can apparently score as high as thresholds that many published studies treat as evidence of a good model. In such catchments, a modeller celebrating a value of 0.8 may simply be describing how easy the catchment is to simulate, not how good their model is.

The reason for this geographic lottery lies in hydroclimatic structure. In humid catchments, streamflow is tightly constrained by precipitation, so even a crude model that converts rain to runoff in roughly the right proportions will track the observed hydrograph reasonably well. In water-limited catchments, evaporation plays a much larger relative role, and the timing and partitioning of water become far more sensitive to model assumptions and data quality. The new study quantifies this systematically using random forest regression trees trained on twenty climatic, hydrological, and topographic catchment attributes. The most important predictors of both lower and upper benchmark performance turned out to be the runoff ratio, aridity, mean flow, and, for the upper benchmark, high-flow magnitudes. Performance generally rose with wetter conditions and fell with increasing aridity, although individual correlations remained weak, underscoring that no single attribute can substitute for a locally computed benchmark.

Computing the lower benchmarks required careful methodological groundwork, and the study delivers practical guidance on every step. The team used the HBV model, a classic bucket-type rainfall-runoff model with thirteen free parameters, applied in a semi-distributed fashion with 200-metre elevation bands to capture the altitude dependence of snow and soil processes. For each catchment, they generated 100,000 random parameter sets, ran the model with each, and evaluated the resulting simulations using three performance measures: the Nash-Sutcliffe efficiency, the Kling-Gupta efficiency, and the non-parametric Kling-Gupta efficiency. A key question was how many random parameter sets are actually needed. By drawing subsets of increasing size, from a single member up to 10,000, and repeating the sampling thirty times, they found that the variability of the lower benchmark shrank steadily and stabilised at around 1000 parameter sets. Their recommendation is therefore straightforward: use 1000 random parameterisations, and no more, to obtain stable lower benchmark values.

How the ensemble should be aggregated proved equally consequential. The researchers compared two approaches: taking the median performance value across all ensemble members, or averaging the simulated streamflow time series themselves and scoring that single ensemble-mean hydrograph. In catchments where simulations were reasonably skilful, the ensemble mean consistently outperformed the median, exceeding it in 94 percent of catchments for the non-parametric efficiency and 99 percent for the Nash-Sutcliffe efficiency. The mean also carries a crucial physical advantage: unlike the median of individual runs, it preserves the water balance. The ensemble mean is not without drawbacks, since it does not correspond to any single physically realisable parameter set, but the authors conclude that it is the stronger and preferable lower benchmark.

The width of the parameter ranges used to generate random values also matters, though less catastrophically than one might fear. The team tested seven range configurations, from roughly half to twice the size of their standard parameter space. Median performance across all catchments varied only modestly, between 0.42 and 0.52, but mean performance increased monotonically with range width, and individual catchments responded in divergent directions, some improving and some degrading as ranges widened. As long as the ranges were not changed drastically, however, benchmark values generally agreed. An alternative approach sidesteps the range question entirely: instead of random parameters, use ensembles of parameter sets calibrated for other catchments in the same region, so-called regional or donated parameter sets. These produced slightly higher, and thus more demanding, lower benchmarks in well-performing catchments, by roughly 0.05 efficiency units, but offered no advantage where random ensembles already yielded low values. The authors suggest regional ensembles may be preferable where available, while random ensembles retain the advantage of requiring no other catchments at all.

The resulting atlas of benchmark values reveals how widely expectations should vary. Across all datasets, the median upper benchmark for the non-parametric efficiency was 0.86, with 90 percent of catchments exceeding 0.65, while the median lower benchmark was 0.68, with 90 percent of catchments exceeding just 0.12 and the most favourable tenth exceeding 0.85. Lower benchmarks varied far more than upper benchmarks, both within and between countries. In the United States, for instance, lower benchmarks climbed close to the upper benchmarks along the east and west coasts, while central regions, where any uninformed model struggles, showed much wider gaps between what is achievable and what is attainable without local information. This pattern means that the difference between the two benchmarks, which can be used to scale a model’s performance into a normalised score between zero and one, largely inherits the geography of the lower benchmark itself.

The implications extend well beyond traditional conceptual models. The authors explicitly note that the benchmarks are suitable for evaluating machine learning approaches, which have swept through hydrology in recent years with claims of dramatic gains over process-based models. Without a local reference frame, it is difficult to know whether a neural network’s impressive score reflects genuine insight into catchment behaviour or simply the fact that the catchment was easy to simulate in the first place. Comparing against the lower benchmark reveals how much skill any model adds over an uninformed baseline, while comparing against the upper benchmark shows how close it comes to the practical ceiling imposed by data quality and model structure. A model can even legitimately exceed the upper benchmark, since the calibrated HBV model used here is not the absolute best possible, and doing so may indicate either beneficial flexibility or a genuinely better representation of the catchment’s hydrology.

The team has released the benchmark values for every catchment, along with the trained random forest models, allowing other researchers to estimate expected lower and upper benchmarks for catchments not included in the original datasets. They intend to update the values as new large-sample datasets appear. The benchmarks are not a perfect instrument: they are tied to runoff alone, they depend on the model and settings used, and when the two benchmarks sit very close together, scaled performance scores can become unstable, demanding careful interpretation of whether both values are high because the catchment is easy or low because the data or the model are inadequate. But the core message is difficult to dispute. A perfect fit is unattainable in practice, an uninformed model can be surprisingly competent, and the gap between those two realities differs from catchment to catchment. Judging every model against the same fixed number, the authors argue, should finally give way to a practice long overdue in hydrology: setting the bar where the landscape, the climate, and the data actually place it.

Subject of Research: Benchmarking hydrological model performance across global large-sample catchment datasets

Article Title: Setting the bar: benchmarks for model performances in large-sample hydrology

Article References: Setting the bar: benchmarks for model performances in large-sample hydrology. (n.d.). https://doi.org/10.5194/hess-30-5857-2026

Image Credits: AI Generated

DOI: 10.5194/hess-30-5857-2026

Keywords: hydrology, rainfall-runoff modelling, HBV model, Nash-Sutcliffe efficiency, Kling-Gupta efficiency, CAMELS datasets, model benchmarks, streamflow simulation, large-sample hydrology, catchment attributes, random forest, model calibration