Simulation study compares how well imputation methods handle missing data under structural bias

Missing data are a persistent problem in observational research, and a new simulation study published in BMC Medical Research Methodology examines how four common strategies for handling them perform when structural bias is present in the data-generating process.

The study, authored by Yulin Gong, Mina Habib, Onyebuchi A. Arah and Roch A. Nianogo, evaluated complete case analysis (CCA) alongside three imputation methods available in R: multiple imputation by chained equations (mice), additive regression with bootstrapping and predictive mean matching (aregImpute), and single imputation by chained random forests (missRanger).

The researchers simulated two datasets under three missingness mechanisms: missing completely at random (MCAR), missing at random (MAR), and missing not at random (MNAR). For each scenario they generated three missing-data proportions of 10%, 25% and 50%, and four sample sizes of 300, 600, 1200 and 5000 observations.

Performance was assessed on the log odds-ratio scale using relative bias and root-mean-square error, together with confidence-interval coverage and width, standard-error calibration, and computation time. This allowed the authors to compare not only how accurate each method’s point estimates were, but also how well each method captured statistical uncertainty.

The results showed that both bias and root mean squared error increased as the proportion of missing data rose, and decreased as sample sizes grew larger.

Complete case analysis and mice generally produced lower bias and lower root mean squared error than aregImpute and missRanger. Interestingly, aregImpute achieved the highest confidence-interval coverage overall, but at the cost of wider confidence intervals and higher root mean squared error, illustrating a trade-off between coverage and precision.

missRanger performed worse than the other methods in the evaluated scenarios. The authors note that no missing-data handling strategy consistently outperformed the others across all settings tested.

Overall, the study concluded that mice provided the most balanced performance across the evaluated scenarios, suggesting that researchers should carefully consider bias-variance trade-offs when choosing an imputation strategy in the presence of structural bias.

As a simulation study, the findings describe performance under the specific conditions the authors generated, including the two simulated datasets, the three missingness mechanisms and the range of missingness proportions and sample sizes studied. The authors present the work as guidance for researchers weighing which missing-data method to apply when structural bias may be present in their data.

Subject of Research: Comparing missing-data handling methods in causal effect estimation under structural bias

Article Title: Performance of different algorithms in dealing with missing data in the presence of structural bias: a simulation study

Article References: Gong, Y., Habib, M., Arah, O. A., & Nianogo, R. A. (2026). Performance of different algorithms in dealing with missing data in the presence of structural bias: a simulation study. BMC Medical Research Methodology. https://doi.org/10.1186/s12874-026-03034-5

Image Credits: AI Generated

DOI: 10.1186/s12874-026-03034-5

Keywords: missing data, multiple imputation, mice, aregImpute, missRanger, complete case analysis, structural bias, simulation study, MCAR, MAR, MNAR, biostatistics