New research exposes critical flaws in some AI models used to predict stroke and diabetes risk, revealing that their underlying datasets lack verified origins and transparency. The study, conducted by a team at Queensland University of Technology (QUT) and the Australian Centre for Health Services Innovation (AusHSI), investigated two highly accessed health datasets hosted on Kaggle, a popular platform for data sharing and machine learning competitions.
Despite being foundational to 125 peer-reviewed studies, these datasets provided negligible information regarding their provenance—where the data originated, how it was collected, and if it involved real patient records. Such gaps in data provenance are alarming, given that some prediction models derived from these datasets have already informed clinical practice, regulatory filings, and numerous systematic reviews.
Applying the TRIPOD+AI reporting framework, recognized internationally for data and model transparency, the researchers found these datasets scored zero out of nine on essential criteria related to data provenance. This finding sharply calls into question the reliability and clinical validity of models trained on them. Lead author Alexander Gibson emphasized the dangers of relying on opaque data sources, warning that “prediction models built on data of unknown provenance have no place in clinical decision-making” and risk misleading healthcare professionals.
The implications are profound in an era where AI-driven clinical tools are rapidly expanding. Unverifiable datasets not only undermine scientific rigor but also threaten patient safety by propagating unreliable diagnostic models. The study urges journals, funding bodies, and data repositories to enforce stricter requirements for dataset transparency to prevent similar issues in the future.
Following these revelations, seven articles utilizing these questionable datasets have been retracted for unreliability, and calls have been made to remove the datasets from Kaggle to avoid further misuse. The findings also contributed to updates in the Collection of Open Science Integrity Guides, advocating for improved standards around data sharing and reporting in AI research.
This investigation highlights a broader challenge—the proliferation of superficially scientific yet fundamentally opaque datasets in health AI research. Without robust provenance documentation and rigorous validation, clinical prediction models risk becoming “fast-churn” research artifacts rather than dependable tools, potentially leading to clinical misjudgments.
As AI technologies become increasingly integrated into healthcare, this study’s warning underscores the necessity of transparency and accountability in dataset curation. Ensuring trustworthy data foundations is paramount to safeguarding the integrity of AI-driven health interventions and ultimately protecting patient outcomes.
Subject of Research: Clinical prediction model data reliability and provenance in AI health research
Article Title: Evidence of unreliable data and poor data provenance in clinical prediction model research and clinical practice
News Publication Date: July 9, 2026
Web References: http://dx.doi.org/10.1186/s12916-026-04981-y
Image Credits: QUT
Keywords: Research ethics, Data fabrication, Data falsification
