Bayesian Framework Merges Clinical Expertise With Machine Learning Diagnosis

Machine learning models can sift through complex biomedical signals with impressive speed, but they often fail in ways that matter in clinics: they behave like black boxes, and they stumble when the patients in front of them look different from the ones they were trained on. A team of researchers from the University of Sheffield and Jiangsu Second Normal University now reports a framework designed to tackle both problems at once, by folding clinicians’ knowledge directly into the diagnostic calculation through Bayesian inference. The approach, called Prior-Informed Machine Learning, or PIML, was tested on three very different diagnostic tasks and generally outperformed standard machine learning models, according to the study published in Frontiers in Bioengineering and Biotechnology.

The central idea is deceptively simple. Instead of letting a machine learning model issue a verdict on its own, the researchers treat the model’s output as one piece of evidence to be weighed against what medicine already knows about the patient. A patient’s colposcopy findings, smear test result, smoking history, or record of previous preterm delivery is converted into a prior probability of disease. Bayes’ Theorem then updates that prior using the machine learning prediction, adjusted by the model’s measured sensitivity and specificity, to produce a posterior probability that a clinician can trace step by step. The authors emphasize that this interpretability refers to the transparency of the updating process itself, not to a guarantee that the resulting probabilities are perfectly calibrated.

What distinguishes PIML from earlier attempts to marry Bayesian reasoning with machine learning, the researchers argue, is the way the required ingredients are produced. Traditional Bayesian diagnostic approaches have relied on sensitivity and specificity values fixed in advance, often drawn from published performance figures. PIML instead embeds the optimization of these two quantities directly into the model development pipeline. Classification thresholds are tuned on validation data, starting from the Youden criterion and then progressively lowered to reach at least 70 percent sensitivity while keeping specificity at or above 50 percent, with the threshold then frozen before it ever touches the test set. This links the discriminative power of the data-driven model to the subsequent Bayesian update in a single coherent procedure.

The evidence feeding the machine learning side comes from electrical impedance spectroscopy, a non-invasive technique that measures how biological tissue impedes electrical current across frequencies from 76 Hz to 625 kHz. Because changes in cellular architecture alter tissue impedance, EIS can flag potentially malignant lesions in the cervix and oral cavity, and it has been used to assess cervical properties during pregnancy. Raw impedance spectra are compressed into four physiologically meaningful parameters using the Cole model, fitted by non-linear curve fitting. These high-dimensional, abstract signals are exactly the kind of information that machine learning handles well but clinicians cannot easily read by eye, which is why the authors see the two sources of knowledge as complementary rather than competing.

The first and largest test was cervical neoplasia detection. The primary cohort contained 1,677 women recruited during routine screening, of whom 528 had biopsy-confirmed cervical intraepithelial neoplasia or carcinoma, with colposcopy assessments and smear test results available as prior knowledge. A second cohort of 112 women from Royal Free Hospital in London, including 19 cancer cases, served as an independent external test set. Four classifiers were compared: decision tree, random forest, support vector machine, and multilayer perceptron. Baseline machine learning models reached AUC values between roughly 0.56 and 0.68, while PIML lifted AUC to between 0.79 and 0.85, with relative gains of about 26 to 40 percent depending on the classifier and cohort.

The external cohort delivered a notable result. All model-related quantities, including hyperparameters, prior probabilities, thresholds, and validation-derived sensitivity and specificity, were fixed using the primary cohort alone before evaluation on the London samples, with no further tuning. Performance on that unseen cohort remained strong, and an external calibration analysis using the decision tree classifier yielded a mean Brier score of 0.116 plus or minus 0.010 across 100 repeated experiments, with reasonable agreement between predicted probabilities and observed disease frequencies in several clinical-evidence subgroups, although deviations persisted in small subgroups. The random forest was the one classifier where the simpler feature-concatenation strategy, in which prior variables are simply appended to the model input, edged ahead of PIML in AUC on both cervical cohorts.

The second application, detecting high-risk oral lesions, was far smaller and the authors treat it as a proof of concept. Sixty-two subjects were enrolled, only nine of whom had histopathologically confirmed dysplasia or malignant lesions, with smoking history serving as the prior. Here the picture was mixed and classifier-dependent. PIML raised AUC by roughly 13 percent for the decision tree, 8 percent for the random forest, and 3 percent for the multilayer perceptron, with marked specificity gains in each case, but the support vector machine favored direct feature integration, which reached an AUC of 0.909. Because only nine high-risk patients underpin these estimates, the researchers caution that the results carry substantial uncertainty, particularly for sensitivity, and that the 100 repeated data partitions used in the analysis do not increase the number of independent patients.

The third task, predicting preterm birth, produced the most striking contrast. The cohort comprised 438 pregnant women scanned at mid-gestation, with 62 delivering before 37 weeks, and prior knowledge came from obstetric history, specifically previous preterm delivery. Under a deliberately generic pipeline without task-specific feature engineering, the baseline machine learning models performed at or near chance, with AUCs between 0.45 and 0.50. PIML transformed these results, raising AUC by approximately 41, 52, 43, and 37 percent for the four classifiers respectively, and restoring a balanced sensitivity-specificity profile where the baseline decision tree had flagged nearly everything as positive. A variant called PIML (APP), which drew priors from the publicly available QUiPP clinical risk tool instead of the training data, performed almost as well, showing the framework can absorb externally derived risk information.

The authors are careful about what these results do and do not establish. All three studies were retrospective, and only the cervical task included an independent external cohort; the oral and preterm analyses rely on repeated internal resampling of their source cohorts. The framework also assumes conditional independence between the machine learning evidence and the clinical prior given disease status, an assumption the authors consider clinically plausible when the two sources capture different aspects of the disease, as with EIS, colposcopy, and cytology in the cervix, but which has not been formally statistically verified. Statistical comparisons used paired Wilcoxon signed-rank tests across the repeated partitions, and the resulting p-values reflect consistency of method differences rather than evidence from 100 independent patient cohorts.

Even so, the framework’s flexibility is its most marketable feature. Because the prior is applied at inference time rather than baked into trained weights, a deployed model can be recontextualized for a new hospital, a shifting disease prevalence, or updated clinical tools without retraining, simply by supplying revised prior probabilities. The authors note the approach is not tied to impedance spectroscopy and could accept imaging features, laboratory biomarkers, or combinations of modalities. Whether that promise survives contact with prospective, multicenter validation in larger independent cohorts, which they identify as the necessary next step, remains the decisive open question for a framework that aims to put clinical judgement and machine learning on equal, calculable footing.

Subject of Research: A Bayesian machine learning framework integrating clinical prior knowledge with impedance spectroscopy for medical diagnosis

Article Title: Prior-informed machine learning for clinically interpretable medical diagnosis

Article References: Lin, Z., Walker, D. C., Murdoch, C., Colley, H. E., Anumba, D. O., & Lang, Z.-Q. (2026). Prior-informed machine learning for clinically interpretable medical diagnosis. Frontiers in Bioengineering and Biotechnology, 14, Article 1973664. https://doi.org/10.3389/fbioe.2026.1973664

Image Credits: AI Generated

DOI: 10.3389/fbioe.2026.1973664

Keywords: machine learning, Bayesian inference, medical diagnosis, electrical impedance spectroscopy, clinical decision support, cervical neoplasia, oral lesions, preterm birth, interpretability, prior knowledge, diagnostic accuracy, Frontiers in Bioengineering and Biotechnology