Simple beats deep: linear interpolation outperforms AI models when real-world clinical data goes missing

The study’s central methodological innovation lies in its use of Markov chains to generate missingness patterns that carry both spatial and temporal structure. Rather than deleting values at random, the researchers first estimated transition probabilities between consecutive missingness states directly from three real-world clinical datasets: MIMIC, the widely used open intensive care database; HiRID, a high-resolution ICU dataset; and ICDEP, an intraoperative dataset collected at Charité medical center that includes labels for postoperative delirium. By learning how missingness in one feature at one time point relates to missingness in neighboring features and adjacent time steps, the simulation model can reproduce the clustered, correlated gaps that characterize genuine clinical monitoring data. This stands in contrast to the majority of prior benchmarking efforts, which generated synthetic missingness under the missing-completely-at-random assumption and therefore systematically underestimated the difficulty that real EHR data pose for imputation algorithms.

The distinction between missingness mechanisms matters because clinical measurements rarely disappear independently. Physiological variables that are captured by the same monitoring device, such as systolic and diastolic blood pressure, tend to be recorded or lost together, producing blocks of simultaneous gaps across features. Similarly, a measurement campaign may be suspended for an entire interval, creating temporal runs of missing values. The Markov chain framework captures both tendencies by treating the joint missingness configuration across features and time as a stochastic process whose transitions are learned from observed data. The authors additionally constructed amplified variants of the learned patterns to strengthen missing-at-random and missing-not-at-random effects, allowing them to probe how imputation methods degrade as the dependence of missingness on observed or unobserved values increases.

To establish a fair comparison, the team first produced complete-case datasets by applying two cleaning procedures and discarding residual missing values. These complete records served as ground truth against which imputed values could be scored. Twenty imputation methods were then applied, spanning a broad methodological spectrum: naive mean imputation, last observation carried forward, linear and quadratic interpolation, running medians, statistical approaches such as multiple imputation by chained equations and k-nearest neighbors, and a family of deep learning architectures including spatio-temporal denoising autoencoders, graph-based variants, bidirectional recurrent imputation models, self-attention-based imputation, and continuous-time autoencoders built on neural ordinary differential equations. This breadth is notable because earlier comparative studies frequently omitted simple time-series baselines such as LOCF and linear interpolation when evaluating sophisticated neural models, making it difficult to judge whether added complexity yields practical benefit.

Evaluation went beyond the conventional mean squared error and mean absolute error, which measure pointwise accuracy but are blind to whether the correlational fabric of the data is preserved. The benchmark therefore incorporated the cross-correlation error, which quantifies how well relationships between features are reconstructed, the normalized root-mean-squared error, and the normalized Wasserstein distance, which compares full distributions of imputed and true values rather than only their means. This multi-metric design reflects a growing recognition in the field that an imputation method can achieve low pointwise error while still distorting the covariance structure that downstream models rely upon, or conversely can preserve distributional properties while sacrificing fine-grained accuracy.

The headline finding is a reversal of the expected hierarchy. The working hypothesis, drawn from prior work suggesting that deep spatio-temporal autoencoders should excel, predicted that these models would outperform simple baselines. Instead, linear interpolation proved most robust when the induced missingness carried spatio-temporal structure, and it supported sufficiently strong downstream classification performance. Deep spatio-temporal autoencoders retained an advantage only when missingness lacked such structure, a regime that arguably resembles the artificial conditions under which many earlier benchmarks were conducted. In other words, the very property that makes real clinical data challenging, namely correlated gaps across sensors and time, appears to neutralize the theoretical advantages of architectures designed to exploit spatial and temporal dependencies jointly.

Several explanations are plausible. Linear interpolation implicitly assumes smooth trajectories between observed points, an assumption that is often reasonable for slowly varying vital signs sampled at high frequency. When gaps are contiguous in time and clustered across features, interpolation within a gap benefits from observed anchors at both boundaries, and the smoothness prior aligns with the physiological dynamics of heart rate, blood pressure, and oxygen saturation. Deep models, by comparison, must learn these dynamics from training data and may struggle when the training distribution of missingness patterns differs from the evaluation distribution, or when the learned representations overfit idiosyncratic correlations. The result echoes earlier reports that k-nearest-neighbor and chained-equation methods frequently match or exceed deep learning approaches on tabular clinical data, and extends that conclusion into the time-series domain under more realistic missingness conditions.

The downstream experiments reinforce the practical stakes. For each dataset the authors defined a clinically meaningful prediction task, such as detecting postoperative delirium in the ICDEP cohort, and trained classifiers on imputed data. The observation that linear interpolation contributed to sufficient downstream performance suggests that elaborate imputation may be unnecessary, and potentially counterproductive, when the goal is predictive modeling rather than faithful reconstruction of individual trajectories. This decoupling of imputation quality, as measured by reconstruction metrics, from downstream utility is itself an important lesson: a method that scores worse on mean squared error may still yield classifiers of comparable or better discrimination, because the classifier can learn to be robust to the artifacts a simple imputer introduces.

For the research community, the open benchmark offers a reusable infrastructure. Because the missingness simulation is parameterized by Markov chains fitted to real data, other groups can generate matched missingness scenarios across datasets, enabling like-for-like comparisons of future methods. The inclusion of spatio-temporal fidelity metrics provides a template for evaluating not just whether values are close to the truth but whether the relational structure of the data survives imputation. This addresses a persistent weakness in the literature, where heterogeneous evaluation setups have produced conflicting conclusions about the merits of deep imputation architectures.

Clinical implications follow naturally. Hospitals deploying machine learning on routinely collected EHR data may achieve reliable results with straightforward interpolation pipelines, reserving computational budgets and validation effort for the downstream task itself rather than for elaborate imputation machinery. The findings also caution against extrapolating benchmark results obtained under missing-completely-at-random simulations to operational settings, where missingness is driven by clinical workflows, sensor reliability, and patient state. As monitoring technology and documentation practices evolve, the learned transition models would need refitting, but the benchmarking framework itself remains applicable, offering a principled path toward imputation choices grounded in the actual missingness structure of the data at hand.

Subject of Research: Benchmarking imputation methods for clinical time series with simulated spatio-temporal missingness

Article Title: Benchmarking imputation methods on real-world clinical time series with simulated spatio-temporal missingness

Article References: Giesa, N., Zhumagambetov, R., Sekutowitcz, M., Piper, S. K., Sikora, V., Balzer, F., Haufe, S., & Boie, S. D. (2026). Benchmarking imputation methods on real-world clinical time series with simulated spatio-temporal missingness. Nature Communications, 17(1), Article 10491. https://doi.org/10.1038/s41467-026-77303-8

Image Credits: AI Generated

DOI: 10.1038/s41467-026-77303-8

Keywords: clinical time series, missing data imputation, deep learning, Markov chains, electronic health records, MIMIC, HIRID, linear interpolation, spatio-temporal missingness, machine learning, benchmarking, autoencoders