Matching, weighting or stratification? Large-scale propensity score methods put to the test

When researchers try to estimate the effect of a treatment using observational healthcare data rather than a randomized trial, they must somehow account for the fact that patients who receive one treatment often differ systematically from those who receive another. Propensity score models are a standard tool for this, and in recent years researchers have increasingly packed very large numbers of covariates into these models, an approach known as large-scale propensity score (LSPS) adjustment. What has remained unclear is whether including so many covariates changes which adjustment strategy — matching, stratification, or inverse probability of treatment weighting — should be applied afterwards.

The team evaluated commonly used adjustment strategies under a new-user cohort design, comparing matching, stratification, and inverse probability of treatment weighting (IPTW) as ways of estimating treatment effects. The evaluation drew on four national healthcare databases and was unusually large in scale: for each model specification, the researchers generated 3,840 treatment effect estimates, covering 24 different treatment cohort comparisons against 160 negative control outcomes.

The authors assessed each adjustment strategy on several performance dimensions: covariate balance between the treatment cohorts, type-I and type-II error, mean-squared error, and the precision of the adjusted effect estimate. These are the operating characteristics that determine whether a strategy is likely to produce reliable conclusions in practice.

Both achieved strong covariate balance and low type-I error, meaning they were less prone to falsely declaring a treatment effect where none exists. Notably, the study also found that no single strategy is clearly superior in all settings, which cautions against a one-size-fits-all approach to large-scale propensity score adjustment.

Based on these findings, the authors offer practical recommendations. They suggest that IPTW with Crump trimming and 1:1 matching are strong default choices for large-scale propensity score adjustment, but that strategy selection should be guided by the specific study context and validated with balance and error diagnostics. They also recommend incorporating the top-performing strategies at least as sensitivity analyses, even when researchers expect a different strategy to excel based on their study’s anticipated operating characteristics, and they broadly recommend the use of empirical calibration to reduce sensitivity to model choice.

Its message is twofold: some strategies are safer defaults than others, but diagnostics — covariate balance, error rates and empirical calibration — should accompany any choice, because the best-performing strategy may vary from one study setting to the next.

Subject of Research: Comparison of large-scale propensity score adjustment strategies in observational studies

Article Title: Real-world performance of large-scale propensity score adjustment strategies: matching, weighting, and stratification

Article References: Li, K. M., Schuemie, M. J., Ryan, P. B., Zhang, L., Chen, Y., Priyam, K., Pratt, N., Hripcsak, G., & Suchard, M. A. (2026). Real-world performance of large-scale propensity score adjustment strategies: matching, weighting, and stratification. BMC Medical Research Methodology. https://doi.org/10.1186/s12874-026-03040-7

Image Credits: AI Generated

DOI: 10.1186/s12874-026-03040-7

Keywords: propensity score, observational studies, confounding, IPTW, matching, stratification, Crump trimming, empirical calibration, healthcare databases, negative control outcomes, type-I error, covariate balance