Machine learning disease prediction models rely heavily on one interpretability method, review finds

Machine learning models are increasingly used to predict whether a disease is currently present or will occur in the future, but their complexity often makes it hard to see why a model reached a particular prediction. A systematic review published in BMC Medical Research Methodology examined how researchers working with observational epidemiological data apply interpretability methods to such models, and found that while the methods are widely used, evaluation and reporting of the explanations remain inconsistent.

The review team, led by Iqbal Madakkatel with Kitty Pham and Elina Hyppönen, searched PubMed, MEDLINE, bioRxiv and medRxiv for primary studies that developed or validated machine learning models to predict disease outcomes in non-clinical populations using structured data on potential risk factors. Studies conducted in care settings, in populations with pre-existing conditions, or using unstructured data were excluded, as were studies focused on intermediary outcomes, COVID-19, mortality, prognosis, or hospital and surgical endpoints. From 2,123 identified records, 1,345 were screened after duplicate removal, and 82 studies were deemed eligible. The review protocol was registered in PROSPERO and conducted according to PRISMA guidelines, and risk of bias was assessed with a tool tailored to interpretability.

The eligible studies used 24 different interpretability methods, but the distribution was heavily skewed. SHAP (SHapley Additive exPlanations) was the most common, appearing in 44 of 82 studies (54%), followed by built-in model feature importance (20 studies, 24%), permutation feature importance (11 studies, 13%), partial dependence plots (5 studies, 6%) and LIME (Local Interpretable Model-agnostic Explanations) (4 studies, 5%). Rule-based approaches appeared in two studies and logistic regression coefficients or odds ratios in three. Most studies provided only global explanations (62 of 82, 76%), and presentation was predominantly graphical (77 studies, 94%), with tables used in 32 studies (39%).

The review also identified substantial gaps in how thoroughly the explanations themselves were evaluated. Implementation of interpretability methods in prototype or real-world tools was reported in only seven studies (9%). Calibration of predicted probabilities was assessed in 26 studies (32%). Only nine studies (11%) met all seven risk-of-bias criteria. Explanation stability was assessed in 32 of the 44 studies for which a clear assessment was possible (73%), with the remaining 38 studies classified as unclear regarding explanation stability. Subgroup or bias-related variation in explanations was assessed in 17 of the 63 studies for which clear assessment was possible (27%), with the remaining 19 classified as unclear.

Based on these findings, the authors recommend standardised interpretability reporting, calibration of models when predicted probabilities are intended to represent absolute risk, application of multiple interpretability methods, routine subgroup and fairness analyses, and quantification of explanation variability. The review’s scope is limited to observational epidemiological studies in populations unselected for existing disease, using structured data on potential risk factors, so its conclusions apply to that setting rather than to clinical prediction models more broadly. Within that scope, the message is that interpretability methods have become a routine part of machine learning-based disease prediction, but the practices surrounding them — how explanations are tested, reported and translated into tools — have not kept pace with their adoption.

Subject of Research: Use of interpretability methods in machine learning models for disease prediction in observational epidemiological studies

Article Title: The use of interpretability methods in machine learning models for predicting current disease presence or future disease occurrence using structured data on potential risk factors: a systematic review of observational epidemiological studies in populations unselected for existing disease

Article References: Madakkatel, I., Pham, K., & Hyppӧnen, E. (2026). The use of interpretability methods in machine learning models for predicting current disease presence or future disease occurrence using structured data on potential risk factors: a systematic review of observational epidemiological studies in populations unselected for existing disease. BMC Medical Research Methodology. https://doi.org/10.1186/s12874-026-03054-1

Image Credits: AI Generated

DOI: 10.1186/s12874-026-03054-1

Keywords: machine learning, interpretability, SHAP, systematic review, disease prediction, epidemiology, risk factors, model calibration, explainability, BMC Medical Research Methodology, methods, machine