Machine learning crash severity review finds imbalance and interpretability gaps

Machine learning has become the dominant tool for predicting how severe a traffic crash will be, but a new systematic review of 306 studies finds that the field’s most popular models are often built on shaky foundations.

The scale of the problem is stark. Of the 306 studies included in the review, 191 papers, or 62.4 percent, did nothing to address class imbalance, the situation in which fatal and serious crashes make up only a tiny fraction of the records. The datasets surveyed show severe cases accounting for as little as 0.1 percent to 5.38 percent of samples, while minor or no-injury crashes comprise 70 to 99.9 percent. In that setting, a model can post an impressive overall accuracy simply by correctly labeling the flood of minor crashes while failing almost completely on the rare, high-stakes outcomes that matter most for emergency response and safety planning.

Among the 115 papers that did tackle imbalance, resampling was the workhorse, used in 84 studies. These techniques include undersampling the majority class, oversampling the minority class, and hybrid approaches. More sophisticated variants such as the Synthetic Minority Over-sampling Technique, Borderline-SMOTE, and Adaptive Synthetic Sampling generate new minority examples in an informed way rather than duplicating existing ones. Ensemble learning appeared in 20 papers, often combining gradient boosting with bagging and random undersampling, and loss-function approaches such as class weighting, hard example mining, and focal loss were used in 11 studies. Cost-sensitive options built into popular tree-based tools like XGBoost and LightGBM allow this kind of training without extensive manual modification.

Interpretability fared somewhat better but still leaves a substantial gap. About 61.8 percent of the reviewed studies, 189 papers, explicitly incorporated some form of interpretability analysis, while 38.2 percent offered none. Among those that did, SHAP, a game-theoretic method that assigns each feature a contribution value for a given prediction, led with 62 papers, followed by global sensitivity analysis in 53, partial dependence plots in 50, local sensitivity analysis in 13, LIME in 7, individual conditional expectation plots in 3, and attention heatmaps in a single study. The authors argue that without such transparency, even a highly accurate model may fail to earn the trust of transportation authorities, since decision-makers need to understand which factors drive a severe-injury prediction before they can design targeted interventions.

The review also documents a problem that sounds mundane but has far-reaching consequences: nobody agrees on what injury severity means. The authors identified four distinct classification approaches across the literature. Some studies simply adopt the pre-defined categories of official crash databases, which vary by jurisdiction; a Rome study, for example, used the city’s own four-level scheme. Others rely on qualitative medical descriptions, such as a New Zealand classification of fatal, serious injury involving hospitalization, and minor injury, or on standardized quantitative metrics like the Injury Severity Score computed from Abbreviated Injury Scale ratings of the three most injured body regions. A third approach defines severity by physical crash parameters such as the change in velocity a road user experiences, and a fourth collapses multiple classes into binary labels to simplify modeling in the face of sparse severe cases.

This definitional patchwork means the same severity level can correspond to different physical outcomes in different papers, which undermines direct comparison of model performance and the generalizability of conclusions. The authors note that merging categories to alleviate data sparsity can introduce subjectivity and obscure critical distinctions between injury levels, and they call for a more standardized framework as a prerequisite for comparable research.

On the modeling side, the review traces a clear evolution. Traditional machine learning methods, including support vector machines, decision trees, feed-forward neural networks, Bayesian networks, clustering, regression, and ensembles, each carry characteristic trade-offs. Support vector machines handle complex nonlinear classification well but cannot inherently rank explanatory variables and become computationally expensive on large datasets. Decision trees are efficient and robust against overfitting but sensitive to imbalanced data. Regression methods are highly interpretable but struggle with nonlinear interactions without manual feature engineering. Deep learning brought convolutional, recurrent, and attention-based architectures, hybrid designs, and more recently large language models that can extract semantic information from unstructured crash narratives and align them with structured roadway and driver data, though these come with high computational overhead, susceptibility to hallucination, and dependence on careful prompt engineering. Generative models such as conditional generative adversarial networks can synthesize rare crash samples to augment imbalanced datasets, but face training instability and mode collapse.

The text mining analysis quantified how research attention has moved. Keyword burst analysis over 2009 to 2025 shows that before 2019 the field centered on fundamentals such as artificial intelligence and statistical methods, while after 2019 hotspots shifted toward specific applications like motorcycle crashes and road traffic injuries, rigorous evaluation terms like prediction accuracy and likelihood ratio tests, and explainability terms including Shapley Additive Explanations and model interpretability. TF-IDF analysis of abstracts, which weights terms that are frequent in a small subset of documents but rare across the corpus, ranked machine learning highest and elevated specialized technique names like random forest, support vector machine, and deep learning, confirming that researchers increasingly name their exact tools. Keyword co-occurrence networks built with CiteSpace contained 284 nodes and 488 edges, with a modularity of 0.7992 and a weighted average silhouette of 0.9184, supporting reliable thematic clustering into ten major groups ranging from crash severity and deep learning to Bayes methods and feature importance.

The literature base itself was assembled rigorously. A search across ACM, Elsevier, IEEE-Wiley, Nature, Web of Science, and Springer databases covering 2005 to 2025 initially returned 4,831 publications. After removing duplicates, 3,677 records remained; title and abstract screening excluded 1,696 irrelevant studies, and full-text evaluation removed another 1,675 that applied machine learning to purposes other than crash severity prediction or were incomplete. The final 306 studies span 2009 to 2025, with publication counts rising notably after 2010 and peaking during 2018 to 2021 at more than 30 papers per year. The datasets behind these studies cover North America, Europe, Oceania, and Asia, from 324 records in a California automated vehicle dataset to more than 730,000 records in a Chicago dataset, spanning intersections, highways, urban streets, and rural roads, and involving pedestrians, cyclists, motorcyclists, motor vehicles, and automated vehicles.

Beyond the two headline gaps, the review identifies a third weakness in evaluation practice. Most studies rely on aggregate metrics such as macro-averaged F1-Score and average AUC, which can mask severe deficiencies on minority classes. The authors advocate per-class recall and F1-Score reporting, wider use of Cohen’s kappa to distinguish genuine skill from chance agreement under imbalanced distributions, and cost-sensitive learning that penalizes misclassifying severe crashes more heavily than minor ones. They also flag limited exploration of emerging methods: severe-injury scarcity is essentially a few-shot learning problem, yet few studies have applied meta-learning or large language models to it, although work from 2023 to 2025 has begun. Real-world deployment faces additional hurdles including data distribution shift, privacy constraints, and limited computational resources, pointing toward continual learning, federated learning, and model compression as future directions. The overall message is that the field should shift from chasing performance metrics toward building models that are balanced across classes, explainable to the authorities that would use them, and evaluated at the granularity where lives are actually at stake.

Subject of Research: A systematic review of machine learning methods for traffic crash injury severity prediction and their methodological gaps

Article Title: Machine learning for traffic crash injury severity modelling: a systematic review integrating text mining analysis

Article References: Chen, J., Meng, X., Chen, Z., Yuan, C., Zhao, Y., Zhang, J., & Hu, H. (2026). Machine learning for traffic crash injury severity modelling: a systematic review integrating text mining analysis. European Transport Research Review, 18(1), Article 96. https://doi.org/10.1186/s12544-026-00856-8

Image Credits: AI Generated

DOI: 10.1186/s12544-026-00856-8

Keywords: machine learning, traffic crashes, injury severity, systematic review, class imbalance, model interpretability, SHAP, deep learning, text mining, PRISMA, road safety, evaluation metrics