AI Flags Hidden Self-Harm History in Europe’s Emergency Rooms

Every year, hundreds of thousands of people arrive at European emergency departments with injuries they inflicted on themselves. For clinicians, one question matters more than almost any other: has this happened before? A documented history of prior intentional self-harm is the strongest single predictor of completed suicide, yet in the chaos of a busy emergency room, that critical piece of information frequently goes unrecorded, uncommunicated, or simply never asked. A new study published in Frontiers in Digital Health suggests that machine learning could help close this dangerous gap, using nothing more than the routine data already collected when a patient walks through the door.

The study draws on the European Injury Database, a multinational surveillance system that collects standardized information on injury presentations at emergency departments across participating countries. Between 2008 and 2023, more than 62,000 attendances for intentional self-harm were recorded from 18 countries, but the researchers faced a fundamental obstacle: the vast majority of records, over 86 percent, carried an unknown value for the outcome they most wanted to understand, namely whether the patient had a documented history of prior self-harm. Rather than discarding this wealth of data, the team set out to understand why the information was missing in the first place, and whether the missingness itself followed a pattern that could be modeled and worked around.

What they found was striking. When they tested how well the missingness of the outcome could be predicted from the available variables, country membership alone explained a remarkable share of the variation, achieving a pseudo-R-squared of 0.537 and an area under the curve of 0.948. In plain terms, knowing which country a record came from was almost sufficient to predict whether the prior self-harm field would be documented at all. This reflects deep structural differences in how national surveillance systems operate, how EDs record clinical information, and how data are transmitted to the central database. Documentation practices, not clinical reality, were driving the gaps.

The analysis also revealed a subtler and more consequential pattern: documentation was positively biased. Across countries, the sparser the overall documentation, the higher the proportion of positive flags among the records that were documented at all, with a Spearman correlation of 0.882. This suggests that clinicians or coders in systems with low documentation rates may only record the prior self-harm field when they have a particularly strong reason to do so, meaning that documented cases are not a random sample of the true prevalence. Any model trained naively on such data would learn to recognize documentation behavior as much as clinical reality.

To navigate these problems, the researchers adopted what they describe as a three-step strategy. First, they excluded three countries whose outcome protocols were structurally absent, meaning the field was never or almost never completed. Second, they redefined the prediction target: instead of trying to predict the true underlying history, the model would predict whether a record carries a documented positive flag versus its absence. Third, they retained records with known outcome status, yielding an analytical dataset of 26,554 records with a prevalence of 13.7 percent. This approach preserved roughly three times as much data as a complete-case analysis would have allowed, which matters because statistical power and the diversity of training examples both shrink rapidly when entire records are discarded.

The modeling approach centered on XGBoost, a gradient boosting algorithm that builds an ensemble of decision trees, each successive tree correcting the errors of those before it. Gradient boosting has become a workhorse of structured clinical data prediction because it handles mixed variable types, captures nonlinear relationships and interactions without manual specification, and performs well on moderately sized datasets. Crucially for a clinical application, the researchers paired the model with SHAP values, a technique rooted in cooperative game theory that assigns each feature a quantified contribution to every individual prediction. This transforms the model from an opaque score generator into something clinicians can inspect, question, and audit.

Performance was solid but appropriately modest. On the held-out test set, the model achieved an area under the receiver operating characteristic curve of 0.800, with a confidence interval running from 0.783 to 0.816. The area under the precision-recall curve was 0.380, a figure that reflects the inherent difficulty of the task given the 13.7 percent prevalence, and the Brier score of 0.100 indicated reasonably well-behaved probability estimates. The model significantly outperformed a logistic regression baseline, with a DeLong test p-value of 3.93 multiplied by ten to the power of minus ten, confirming that the nonlinear structure of the boosted trees captured relationships the linear model could not.

Threshold selection is where statistical performance meets clinical reality. At the optimized operating point of 0.080, the model detected 80.8 percent of documented positives while correctly clearing 62.8 percent of negatives, and the negative predictive value reached 95.4 percent. These trade-offs matter enormously in an emergency setting. A high-sensitivity configuration means fewer at-risk patients slip through, but it also flags many patients who will turn out to have no documented history, generating additional workload. The negative predictive value is particularly relevant for triage: when the model says a patient is unlikely to have a documented prior self-harm flag, it is right more than nineteen times out of twenty.

Recognizing that a model is only useful if it can run at the moment a patient arrives, the researchers built a second version restricted to variables available at time zero, meaning at triage before any extended clinical workup. This time-zero model achieved an AUC of 0.762, only modestly below the full model, and still comfortably ahead of the linear baseline at 0.729. The small performance sacrifice is a worthwhile price for a tool that can actually inform the first clinical encounter rather than arriving after decisions have already been made.

The SHAP analysis pointed to proximal risk factor documentation, place of occurrence, and patient age as the dominant drivers of predictions. This raised an important methodological concern: the proximal risk factor variable could be circular, since documentation of risk factors might itself be correlated with documentation of prior self-harm history, meaning the model could partly be learning documentation habits rather than clinical patterns. To test this, the team reran the analysis with that variable excluded. Performance dipped only slightly, to an AUC of 0.765, confirming that the model’s core signal does not depend on the potentially circular feature.

Perhaps the most methodologically revealing exercise was the country prediction test. When the researchers trained a model to guess which country a record came from using only the clinical features, it achieved 95.3 percent accuracy, fourteen times better than random chance. This demonstrates that the clinical variables in the database encode substantial geographic information, whether through differing coding conventions, injury mechanisms, demographic profiles, or health system characteristics. This implicit geographic encoding offers a plausible explanation for a troubling pattern in the validation results: the model scored 0.800 on the randomly split test set, but only 0.674 under temporal validation and 0.658 under group-aware cross-validation, where entire countries or time periods were held out. The gap between random and grouped validation is a classic signature of leakage, where the model exploits site-specific quirks that will not transfer to new settings.

Calibration, the agreement between predicted probabilities and observed frequencies, showed a similarly nuanced picture. In most countries with sufficient data, the model’s probability estimates were adequately calibrated, meaning a predicted risk of twenty percent roughly corresponded to twenty percent of such patients having a documented flag. In low-volume countries, however, calibration broke down, likely because the model had fewer examples to learn country-specific patterns and because local documentation practices differed from the training distribution. This is a familiar challenge in multinational health data: a model that is well calibrated on average can be badly miscalibrated for specific subpopulations.

The fairness analysis surfaced what the authors identify as a critical limitation. Sensitivity was substantially lower in older adults, at 68.0 percent, with an AUC in that subgroup of 0.722 and a wide confidence interval stretching from 0.617 to 0.822. Older patients who self-harm are a population of particular concern, since late-life self-harm is associated with especially high risk of completed suicide, often with higher lethality of methods and less prior contact with mental health services. A model that under-detects documented history in this group could inadvertently direct attention away from some of the patients who need it most. The authors are candid that this disparity must be addressed before any deployment.

The authors are equally careful about what the model is and is not doing. This is a retrospective pattern-recognition task, not a crystal ball. The model does not predict future suicide attempts, and it does not infer undocumented clinical history. It learns which combinations of injury characteristics, demographics, and clinical features recorded at ED attendance are associated with a positive prior self-harm flag in the surveillance database. The clinical rationale is that patients whose observable patterns strongly resemble those of patients with documented prior histories may warrant closer attention, but the output is a flag for consideration, not a diagnosis.

The path from these results to clinical practice runs through several gates that the authors spell out. Local validation is essential, because the geographic encoding demonstrated in the country prediction test means a model trained on pooled European data may behave very differently in any single health system. Country-specific recalibration would adjust the model’s probability outputs to match local documentation prevalence and coding practices. And perhaps most importantly, the model would need to be embedded within structured post-identification care pathways, because the evidence from large-scale ED studies is clear that screening alone, whether by questionnaire or algorithm, does not by itself reduce suicidal behavior. Identification only saves lives when it connects to follow-up: safety planning, rapid outpatient linkage, means restriction counseling, and repeated contact.

There are also broader lessons here for the field of clinical machine learning. The study demonstrates that formal missingness analysis should precede model building when outcomes are inconsistently recorded, that the choice between complete-case analysis and target redefinition can multiply usable data several-fold, and that validation must go beyond random splits to include temporal and group-aware designs that expose leakage. The finding that a model can look excellent on a random test set while degrading sharply under realistic validation conditions is a cautionary tale for any predictive tool trained on multinational data. Equally, the demonstration that SHAP-based interpretability can coexist with strong discrimination, and that sensitivity analyses can probe potentially circular features, offers a template for transparent model development in sensitive clinical domains.

For emergency departments weighing whether such tools belong in their workflows, the realistic framing is as a decision support layer rather than an autonomous gatekeeper. A model that flags a presentation as consistent with prior self-harm history could prompt a targeted question at triage, trigger a review of available records, or cue a brief psychosocial assessment, all actions that remain under clinician control. Used this way, even a model with imperfect sensitivity adds value by making the invisible visible at the moment it matters. The study’s contribution is not a finished product but a rigorous demonstration that routine surveillance data, handled with methodological care, can support earlier recognition of one of the highest-risk populations walking through the emergency department door, provided the surrounding systems of care are ready to respond.

Subject of Research: Machine learning prediction of documented prior intentional self-harm history in European emergency department injury surveillance data

Article Title: Predicting documented prior intentional self-harm in emergency departments: an interpretable gradient boosting approach using European injury surveillance data

Article References: Giustini, M., Pitidis, A., Fazio, C., Alves, T., Fian, T., Valkenberg, H., Nijman, S., Papadakaki, M., & Carannante, A. (2026). Predicting documented prior intentional self-harm in emergency departments: an interpretable gradient boosting approach using European injury surveillance data. Frontiers in Digital Health, 8, Article 1964834. https://doi.org/10.3389/fdgth.2026.1964834

Image Credits: AI Generated

DOI: 10.3389/fdgth.2026.1964834

Keywords: machine learning, XGBoost, self-harm, suicide prevention, emergency department, EU-IDB, SHAP, predictive modelling, missing data, algorithmic fairness, injury surveillance, digital health