Registry Studies Rarely Test How Missing Patient Data Could Reverse Their Findings

Patient-reported outcome measures, or PROMs, have become a staple of clinical quality registries, capturing symptoms, function and quality of life directly from patients at low marginal cost across large observational cohorts. But a new methodological audit published in BMC Medical Research Methodology suggests that the statistical machinery used to handle the inevitable gaps in these data has not kept pace with the methods available. In a scoping review of 61 registry-based PROM studies published between 2018 and 2024, not a single study reported a tipping-point analysis in its accessible methods or supplementary material, and only one study reported a delta adjustment, applied merely as a sensitivity check.

The audit, led by Johan Eriksson and reported in accordance with the PRISMA extension for scoping reviews, combined a Nordic-specific PubMed search anchored in registry nomenclature with global searches in Embase, CINAHL and PubMed. Of 425 records identified, 397 remained after deduplication, 82 were retrieved for full-text review, and 61 met all inclusion criteria: 17 from the Nordic search and 44 from the global searches. To qualify, studies had to include an explicit missing-data discussion in their methods or statistical analysis sections, describing a specific analytic approach, a missingness mechanism, or a responder-versus-non-responder comparison as part of the statistical methods.

The findings reveal a two-level gap. At the level of analytic completeness, formal sensitivity analysis for missing-not-at-random data was rare: of the 61 studies, 18 (30%) reported any MNAR sensitivity analysis, formal or informal, but only five used formal MNAR models such as selection models, shared-parameter joint models or pattern-mixture frameworks. At the level of basic transparency, 24 studies (39%) stated no missing-data strategy at all, relying implicitly on complete-case analysis, and only 17 (28%) distinguished never-responders, patients who never return PROMs, from follow-up dropout among baseline responders. Thirty-three studies (54%) compared responders with non-responders, but standardized mean differences were rarely used, with most relying on unadjusted statistical tests or qualitative descriptions.

The distinction between never-responders and dropout matters because the two mechanisms carry different risks. Never-response is typically associated with older age, higher comorbidity burden, lower socioeconomic status and more severe disease, a form of structural non-participation. Dropout, by contrast, often occurs because deteriorating health reduces the capacity or motivation to respond, meaning the missing PROM value may depend on the unobserved outcome itself even after conditioning on observed covariates. That is the definition of a missing-not-at-random mechanism, and it is precisely the situation in which complete-case estimates can be quietly biased.

To close the gap between methodological availability and practical uptake, the authors propose a three-step workflow. Step one is non-response diagnostics: comparing responders and non-responders on prespecified baseline covariates using standardized mean differences, presented separately for baseline non-response and follow-up dropout, with an SMD above 0.10 flagging potentially meaningful imbalance. Step two adopts inverse probability weighting as the default primary analysis, reweighting observed cases by the inverse of their estimated response propensity, with weight truncation at the first and 99th percentiles and robust sandwich standard errors to account for weight-estimation uncertainty. Complete-case and weighted estimates are reported side by side. Step three applies delta adjustment across a grid of assumed deviations from the missing-at-random assumption to identify the tipping point at which the substantive conclusion would change.

The tipping-point framework is what the authors consider its most clinically communicable feature. Rather than asking what the true MNAR mechanism is, an unidentifiable question, it asks how much worse the missing patients would need to be, relative to what the observed data predict, for the study’s conclusion to reverse. Missing PROM values are assumed to be, on average, delta points lower than predicted under MAR, the analysis is repeated across a grid of delta values, and the tipping point is the value at which the estimated treatment effect crosses zero or a prespecified clinical threshold. Ideally the tipping point is judged against the minimally important difference of the instrument in use; the illustrative classification in the paper labels results stable above 10 points, sensitive between 5 and 10, and unstable below 5.

The workflow is demonstrated on a simulated registry-like cohort of 8,000 patients with covariates typical of a Nordic register: age averaging 62 years, three disease stages, a comorbidity count, a socioeconomic status proxy and a binary treatment indicator. Three missingness components were simulated: never-responders comprising roughly 37% of the cohort whose non-response depended on age, socioeconomic status, comorbidity and stage; sporadic baseline non-response; and MNAR dropout at follow-up that depended on both observed covariates and the unobserved follow-up PROM value, with a differential component by treatment group. Only 4,753 patients (59%) provided baseline PROM data, and 3,829 (48%) had complete data at both timepoints.

The diagnostics step proved its worth immediately. For never-response, covariate imbalance was driven by comorbidity, age, disease stage and low socioeconomic status, with treatment balanced. For follow-up dropout among baseline responders, the profile differed sharply: baseline PROM was the strongest correlate, and treatment was now imbalanced, reflecting differential dropout in the treated group. Separating the two mechanisms therefore exposed treatment-associated dropout that a pooled comparison would have concealed, with direct consequences for how the weighting model should be specified. In the primary simulated scenario, with a deliberately sub-clinical true treatment effect of 0.8 PROM points, the complete-case estimate was 1.21 points (95% CI 0.69 to 1.72), the single-stage inverse-probability-weighted estimate was 1.22 (0.69 to 1.74) and the two-stage estimate addressing both non-response and dropout was 1.23 (0.67 to 1.79), a concordance the authors attribute to treatment being balanced with respect to baseline response.

The tipping-point analysis then quantified fragility. Under an asymmetric specification in which missing outcomes in the treatment group were shifted by delta points and control-group missing outcomes by delta divided by three, the point estimate crossed zero at a tipping point of 7.2 PROM points, a result classified as sensitive under the illustrative thresholds. The lower bound of the confidence interval included zero from roughly delta 4.7 onward, meaning statistical significance would be lost before the estimate itself reversed. In a secondary scenario with a clinically meaningful true effect of 4.0 PROM points, the tipping point rose to 30.5 points and the result was classified as stable, illustrating that the workflow produces appropriately robust classifications when the underlying evidence is strong.

The authors are careful about the limits of their audit. Because it examined full-text methods sections from a geographically and temporally bounded sample, it reflects reporting practice rather than what analyses were necessarily performed; analyses may have been conducted without being described, and authors were not contacted to verify, so all findings are framed as not reported rather than not done. The simulated delta specification and the robustness thresholds are explicitly illustrative and should be adapted to the minimally important difference of each instrument, and the workflow does not address missing covariate data. Even so, the practical recommendations are concrete: separate never-responders from dropout, report SMD-based non-response diagnostics, and run at least one tipping-point analysis. These require no new data collection and no specialist software, and the authors argue that adopting such a minimum reporting set would be a substantive and achievable improvement in field standards for registry-based PROM research.

Subject of Research: Missing-data handling and MNAR sensitivity reporting in registry-based patient-reported outcome measure studies

Article Title: A pragmatic workflow for handling missing PROM data in registry studies: a methodological scoping review and simulated worked example

Article References: Eriksson, J. (2026). A pragmatic workflow for handling missing PROM data in registry studies: a methodological scoping review and simulated worked example. BMC Medical Research Methodology, 26(1), Article 225. https://doi.org/10.1186/s12874-026-03058-x

Image Credits: AI Generated

DOI: 10.1186/s12874-026-03058-x

Keywords: PROMs, patient registries, missing data, MNAR, tipping-point analysis, delta adjustment, inverse probability weighting, scoping review, non-response diagnostics, standardized mean differences, simulation study, research methodology