Cause-Specific Hazard Models Help Detect Prediction Errors Under Competing Clinical Risks

In a study published in Diagnostic and Prognostic Research, biostatisticians Sarwar I. Mozumder, Sarah Booth, Richard D. Riley, Mark J. Rutherford, and Paul C. Lambert show that evaluating cause-specific survival curves alongside absolute risks provides an effective method to detect miscalibration during external validation of prognostic models under competing events. When applying prognostic tools to external populations, assessing calibration—the agreement between predicted probabilities and observed outcomes at a specific time point—is essential for predictive accuracy. In competing risks data, patients may experience alternative terminating events before the outcome of primary interest occurs. Consequently, calculating real-world cause-specific absolute risks requires modeling multiple competing endpoints simultaneously. The authors establish that because each cause-specific absolute risk prediction incorporates parameters from every competing event model, miscalibration detected in external data cannot be accurately localized without independently assessing the underlying cause-specific hazards models.

The authors emphasize that two main modeling frameworks exist for predicting absolute risk under competing events: subdistribution hazards modeling and the cause-specific hazards approach. While previous validation literature predominantly focused on the Fine and Gray subdistribution hazards formulation to evaluate calibration on a single isolated event, clinical decision-making often demands understanding risks across all competing causes. For example, to assess the clinical impact of cardiotoxic cancer treatments, oncologists and patients benefit from understanding cardiovascular mortality risks alongside cancer-specific mortality. Decomposing overall mortality into distinct cause-specific components provides greater clinical insight. Under the cause-specific hazards framework, separate models are developed for each competing event, which makes it straightforward to isolate covariate effects on specific causes rather than dealing with subdistribution models where regression coefficients reflect composite effects across all competing risks.

To formalize their methodology, the authors detail the mathematical relationships connecting cause-specific hazards to real-world absolute risks. For an individual facing multiple competing events, the cause-specific hazard represents the instantaneous rate of dying from a specific cause given survival up to that moment. The cause-specific absolute risk, also termed the cause-specific cumulative incidence function, is obtained by integrating the product of the all-cause survival function and the specific cause-specific hazard over time. The all-cause survival function represents the negative exponential of the cumulative integral of all cause-specific hazards across all competing endpoints. Nonparametric calculation of cause-specific absolute risk in sample data is achieved using the Aalen-Johansen estimator, which weights increments in cause-specific event proportions by the prior all-cause Kaplan-Meier survival estimate, ensuring the sum of all cause-specific absolute risks equals the total all-cause risk.

When estimating covariate effects on cause-specific hazards, researchers frequently implement Cox proportional hazards models or Royston-Parmar flexible parametric survival models. While the Cox model estimates log-hazard ratios effectively, its baseline hazard function is not directly modeled, making absolute risk predictions dependent on post-hoc nonparametric tools such as the Breslow estimator. In contrast, Royston-Parmar flexible parametric survival models explicitly formulate the baseline cumulative hazard on the log-time scale using restricted cubic splines. This parametric specification enables analytical derivation of the cause-specific survival function without numerical integration. In addition, the flexible parametric framework accommodates non-proportional hazards by incorporating interaction terms between covariates and spline functions of time. This flexibility allows investigators to generate individual cause-specific absolute risk estimates and composite all-cause survival predictions across complex follow-up periods in validation cohorts.

The core methodological proposal advanced by the authors is that external validation should include separate calibration evaluations using the complement of the cause-specific survival function. Because an absolute risk estimate combines hazard parameters across every competing cause, miscalibration in the predicted absolute risk of one event can be induced by errors in that specific model, errors in competing risk models, or combined structural shifts. Evaluating the complement of the cause-specific survival function—one minus the cause-specific survival probability—isolates components derived exclusively from an individual cause-specific hazards model. This diagnostic step restores a direct link between model coefficients and event probabilities, enabling researchers to pinpoint precisely which regression model is mis-specified or suffers from an altered baseline hazard when evaluated in external populations. Consequently, visual inspection of these isolated curves prevents unnecessary recalibration of correctly specified sub-models.

Assessing the complement of the cause-specific survival function requires careful handling of dependent censoring, as patients experiencing competing events are treated as censored within cause-specific hazard models. Because competing outcomes often share risk factors, such as age or chronic conditions, the assumption of completely independent censoring is untestable and frequently violated. To circumvent grouping individual predictions into broad risk percentiles, the authors calculate jack-knife pseudo-observations, which replace incomplete survival data with complete individual values. To estimate pseudo-observations for one minus cause-specific survival accurately, the authors categorize the linear predictor into several groups and apply a mixture Kaplan-Meier estimator. Stratifying across the prognostic index weakens the independent censoring requirement into a more plausible conditionally independent censoring assumption, provided the linear predictor adequately reflects the underlying dependency structure. This adjustment ensures that observed risks generated through pseudo-observations remain reliable across the validation cohort.

To generate continuous calibration curves across individual predicted probabilities, the authors model the calculated pseudo-observations using generalized linear models equipped with restricted cubic splines. Unlike loess smoothers or nearest-neighbor estimators that require selecting arbitrary bandwidth parameters, restricted cubic splines provide smooth curves through predefined knot locations. Applying a logit link function ensures that the resulting calibration curve remains constrained between zero and one across the entire predicted risk spectrum. The researchers place boundary knots at the fifth and ninety-fifth percentiles of the predicted risk distribution, which stabilizes curve estimation and prevents distortion from sparse observations in the distribution tails. To aid visual interpretation, the authors supplement the smoothed curve with observed risks grouped across ten equal risk intervals, offering clinicians both continuous curves and grouped summaries on traditional calibration plots. This combined visual format makes model deviations readily apparent across both central and extreme risk ranges.

Alongside graphical calibration curves, the authors recommend reporting summary calibration statistics and overall predictive performance measures. In an ideal model, the calibration slope equals one, whereas values differing from one signify overfitted or narrow predictions, and calibration-in-the-large equals zero, indicating perfect average calibration. To capture overall predictive accuracy, the authors evaluate the Brier score, which incorporates inverse probability of censoring weights derived from flexible parametric models to account for dependent censoring. Because the Brier score magnitude depends heavily on underlying event rates, the authors emphasize the Index of Prediction Accuracy, which scales the model Brier score against a null benchmark model fitted without covariates. An Index of Prediction Accuracy of one indicates perfect prediction, zero represents an uninformative model, and values below zero identify models that perform worse than assigning average baseline risk. These metrics provide rigorous numerical benchmarks that supplement qualitative visual impressions from calibration curves.

To demonstrate the practical merits of their framework, the researchers conducted simulation studies following the cause-specific data-generating approach developed by Beyersmann and colleagues. They simulated a derivation cohort of fifty thousand individuals to minimize sampling variation and establish baseline model parameters. The data-generating mechanism incorporated continuous predictors with non-linear functional forms, a four-level categorical variable, and proportional hazards, with administrative censoring fixed at ten years. The investigators subsequently generated three independent external validation cohorts, each containing fifty thousand individuals, to replicate distinct clinical validation challenges. Scenario one simulated an elevated baseline cause-specific absolute risk for the competing event, mimicking populations with higher comorbidity burdens. Scenario two modeled a shift in the baseline cause-specific hazard for the primary outcome. Scenario three combined baseline hazard changes for both the primary and competing causes, reflecting external populations where multiple event distributions diverge simultaneously.

The simulation findings demonstrated that assessing cause-specific model components successfully identifies model misspecifications that remain undetected when viewing absolute risk plots alone. In scenario one, where only the competing event hazard changed, absolute risk calibration plots showed apparent miscalibration for all causes. However, inspecting the complement of cause-specific survival curves revealed that the primary cause model remained well-calibrated, while the competing event model exhibited marked failure. In scenario two, when the baseline hazard for the primary outcome shifted, the absolute risk calibration slope for that cause equaled one, but the calibration-in-the-large statistic reached 0.55, illustrating that a correct slope can mask substantial baseline hazard miscalibration. In scenario three, with shifts across both causes, absolute risk calibration appeared deceptively acceptable visually for cause one, but cause-specific model curves revealed calibration slopes exceeding one for both outcomes, correctly signaling narrow predictions and incorrect baseline hazard magnitudes.

To enable researchers to implement these diagnostics in practical validation workflows, the authors released a dedicated Stata command, calplotcr, available publicly on GitHub as an extension of existing survival packages. In their discussion, the authors call for updating clinical model reporting guidelines, recommending that developers publish cause-specific baseline hazard values across multiple prediction time points alongside all regression coefficients, extending standard TRIPOD recommendations. The researchers acknowledge limitations, noting that the conditionally independent censoring assumption cannot be empirically verified in external cohorts and that unmeasured confounding may persist. Nonetheless, conditioning on available predictors through mixture Kaplan-Meier stratification or inverse probability weighting minimizes potential bias. By evaluating the complement of cause-specific survival alongside absolute risks, biostatisticians can diagnose the exact origin of predictive failures, ensuring clinical risk models remain well-calibrated when deployed in diverse patient populations.

Subject of Research: External validation and calibration methods for cause-specific hazard prediction models under competing risks

Article Title: Calibration of cause-specific absolute risk for external validation using each cause-specific hazards model in the presence of competing events

Article References: Mozumder, S. I., Booth, S., Riley, R. D., Rutherford, M. J., & Lambert, P. C. (2025). Calibration of cause-specific absolute risk for external validation using each cause-specific hazards model in the presence of competing events. Diagnostic and Prognostic Research, 9(1), Article 23. https://doi.org/10.1186/s41512-025-00197-5

Image Credits: AI Generated

DOI: 10.1186/s41512-025-00197-5

Keywords: Cause-specific hazards, Competing risks, Absolute risk, External validation, Prognostic models, Calibration slope, Calibration-in-the-large, Pseudo-observations, Restricted cubic splines, Brier score, Index of Prediction Accuracy, Flexible parametric survival models, Dependent censoring, Inverse probability censoring weights, Aalen-Johansen estimator, Kaplan-Meier estimator, calplotcr