When Flood Maps Lie: Smarter AI Brings Honesty to Bangladesh Disaster Prediction

When catastrophic floods strike, the maps that guide evacuations, land-use zoning and infrastructure investment are often produced by machine-learning models that look impressively accurate on paper. A new study of the devastating August 2024 flash flood in the Feni district of south-eastern Bangladesh argues that many of those impressive numbers are, in a precise technical sense, too good to be true. The research, published in Discover Geoscience, dissects the standard recipe used across flood susceptibility mapping and finds three hidden failures: inflated accuracy from spatial leakage, hydrologically backwards relationships buried inside the model, and probabilities that carry no honest measure of uncertainty. The authors then assemble a framework that fixes all three without sacrificing predictive skill.

The Feni disaster was among the most destructive in the region’s recent history. Between 19 and 24 August 2024, exceptional monsoon rainfall over the Feni, Muhuri and Kahua river basins, compounded by trans-boundary inflows, inundated low-lying deltaic terrain within hours and disrupted the lives of hundreds of thousands of residents. The observed flood mask shows that roughly 21 percent of the analysed land pixels were underwater. For the researchers, based at Jahangirnagar University and BRAC University in Dhaka, the event offered a rigorous testbed: eleven conditioning factors spanning topography, hydrology, land surface and rainfall forcing were generated on Google Earth Engine as co-registered 30-metre rasters, paired with a balanced inventory of 1,510 sampled points, covering 982,866 mapped pixels in total.

The first problem the study quantifies is spatial leakage. Because flood labels and terrain features are strongly autocorrelated in space, a conventional random train-test split places near-duplicate neighbours on both sides of the partition; the inventory contains pairs of points as close as a single 30-metre pixel, with 720 pairs within 60 metres. When the same models were evaluated with random splits, ROC-AUC soared to about 0.998 for every learner. Under spatially blocked cross-validation, in which the region is divided into five k-means clusters and entire blocks are held out, the honest figure for the best model was 0.938, an inflation of roughly 0.06 to 0.09 AUC. The gap proved structural: repeating the comparison with three, five, eight and ten blocks left a persistent gap, and replacing coordinate blocks with hydrological sub-basins produced an even larger optimism of 0.089, since contiguous basins are a harsher extrapolation test.

The second failure is subtler and arguably more alarming. An unconstrained gradient-boosting model can achieve excellent discrimination while encoding relationships that contradict basic hydrology. Audited through partial-dependence analysis, the unconstrained model was monotone in the physically expected direction on only 54.5 percent of its response surface. Most damaging, its topographic wetness index, a measure of how water accumulates in terrain, sometimes became protective against flooding, and its vegetation index sometimes appeared to promote inundation. In other words, the black box had learned, from noisy data, that wetter ground floods less, a relationship that would badly mislead anyone extrapolating the map into unfamiliar terrain or explaining it to emergency planners.

The researchers’ remedy is to inject hydrological knowledge as hard monotonicity constraints during model training. Each factor receives a scientifically defensible sign: susceptibility must rise with rainfall and wetness, and fall with elevation, slope, ruggedness and distance to rivers. Where the hydrological effect is genuinely contested, such as curvature, no constraint is imposed and the data are audited afterwards. The constrained gradient-boosting implementation enforces the ordering through constrained split selection and leaf-value clipping, so no accessible input perturbation can produce a hydrologically backwards prediction. Crucially, this physical guarantee came at no measurable cost in skill: the constrained model matched its unconstrained counterpart on every metric, achieving a spatial AUC of 0.938 and a Matthews correlation coefficient of 0.700, while actually improving the Brier score slightly.

To make physical accountability routine rather than optional, the team introduced a Physical Consistency Score, which repurposes partial-dependence analysis from description to verification. The score measures the fraction of response increments across all constrained factors whose sign matches the hydrological prior; the constrained model scores 100 percent by construction, while the unconstrained model’s 54.5 percent quantifies exactly how much of its surface contradicts hydrology. The authors then went further, testing the priors themselves against reality: binning the observed 2024 inundation by each factor, the empirical flood frequency trended in the same direction as every imposed prior, with Spearman correlations reaching as high as minus 1.00 for slope and distance to river and plus 0.99 for wetness. The constraints, in other words, reproduce rather than merely assert the directional structure of the actual flood.

The third pillar addresses probability and uncertainty. Raw boosting scores are typically over-confident, and the study found an expected calibration error of 0.150 before correction. Cross-fitted isotonic regression, fitted on out-of-fold spatial predictions, reduced that error roughly tenfold to 0.016 and improved the Brier score from 0.155 to 0.095, so that a mapped value near 0.7 now corresponds to an observed flood frequency near 0.7 within the inventory design. On top of the calibrated scores, the researchers wrapped a conformal prediction layer, using spatially stratified Mondrian calibration to supply distribution-free uncertainty. Because genuine spatial extrapolation violates the exchangeability assumption underlying conformal guarantees, the layer is as much diagnostic as corrective: marginal coverage held at 0.87 against a 0.90 target, but worst-region coverage fell to 0.48, an honest exposure of where the map can and cannot be trusted.

The final products are unusually transparent for the field. The five-class susceptibility map concentrates the highest classes along river corridors and the lowest-lying tracts, and the conformal decision map partitions the region into confident high-risk zones covering 29 percent of the area, confident low-risk zones at 6 percent, and a large precautionary zone at 65 percent where additional data or caution are warranted. Validated against the spatially independent observed inundation extent, the calibrated susceptibility attained an AUC of 0.75, with observed flood frequency rising strictly across predicted classes from 1.0 percent to 45.4 percent. Two-thirds of the observed inundation fell within the High and Very-High classes, which occupy 40 percent of the area. The learned map also outperformed every purely physical benchmark, including topographic wetness indices and elevation-based composites, the best of which reached only 0.73.

The authors are careful about the scope of their claims. The validation observations derive from the same August 2024 event, so transferability to other floods, regions and hydroclimatic conditions remains untested; the map is an event-conditioned diagnosis rather than a return-period product, and the balanced inventory makes the scores relative rather than absolute probabilities. Validation covers binary inundation extent only, not flood depth, velocity or duration, and the absence of channel bathymetry and discharge hydrographs precludes direct comparison with full hydrodynamic models such as HEC-RAS or LISFLOOD-FP. Sensitivity analyses over the constraint set, the number of spatial blocks, the choice of calibrator and even the resolution of the terrain data, which was coarsened to 120 metres with results holding within a narrow band, leave the conclusions unchanged.

The broader message resonates well beyond Bangladesh. Flood susceptibility mapping, the study argues, does not primarily need another gradient-boosting benchmark; it needs maps that are physically accountable, probabilistically honest and explicit about where they can be trusted. A conventional boosting-plus-SHAP analysis of this dataset would have reported an AUC near 0.998, a feature-importance chart and a declaration of success, while concealing nine points of leakage-driven optimism, a wetness relationship running the wrong way on most of its range, and mis-calibrated probabilities with no uncertainty attached. By reframing the task around hard physical constraints, calibrated probability and spatially stratified conformal uncertainty, all achieved with existing data and reproducible code, the framework offers a practical template for turning flashy machine-learning scores into decision-grade tools for the places that need them most.

Subject of Research: Physics-constrained and uncertainty-aware machine learning for flood susceptibility mapping of the 2024 Feni flash flood in Bangladesh

Article Title: Physically consistent and spatially conformalized machine learning for trustworthy flood susceptibility mapping of the 2024 Feni flash flood

Article References: Diptho, R. A., Ghosh, P., & Chowdhury, S. H. (2026). Physically consistent and spatially conformalized machine learning for trustworthy flood susceptibility mapping of the 2024 Feni flash flood. Discover Geoscience, 4(1), Article 390. https://doi.org/10.1007/s44288-026-00759-0

Image Credits: AI Generated

DOI: 10.1007/s44288-026-00759-0

Keywords: flood susceptibility mapping, machine learning, monotonic constraints, conformal prediction, spatial cross-validation, calibration, explainable AI, flash flood, Bangladesh, Google Earth Engine, gradient boosting, trustworthy AI