AI Model Fuses Scans, Blood Tests and Endoscopy Images to Gauge Crohn’s Disease Activity

Assessing how active Crohn’s disease is inside the small bowel has long been an exercise in piecing together fragments of evidence. Blood tests hint at inflammation but cannot localize it. Cross-sectional imaging reveals wall thickening and enhancement but says little about the mucosal surface. Endoscopy shows the lining directly but reaches only as far as the scope can travel. A new study published in BMC Medical Imaging by researchers at the First Affiliated Hospital of Anhui Medical University argues that these fragments are far more powerful when fused, and it demonstrates the point with a multimodal machine learning framework that combines clinical data, radiomic features from computed tomography enterography, and deep learning analysis of double-balloon enteroscopy images to evaluate small bowel Crohn’s disease activity.

The retrospective study, approved by the hospital’s institutional review board with informed consent waived and all patient data anonymized, set out to build models that could classify disease activity in patients with small bowel Crohn’s disease, a form of the condition that is notoriously difficult to monitor because the affected segments often lie beyond the reach of conventional colonoscopy. The team, led by Yi Shen, Chang Rong and Han Zhang from the Department of Radiology together with gastroenterologist Jing Hu and corresponding author Xingwang Wu, drew on three complementary data streams collected as part of routine clinical workup, then asked a simple question with far-reaching implications: does a model that sees everything outperform models that see only one thing?

The first stream was clinical. Using univariate and multivariate logistic regression, the researchers sifted through demographic and laboratory variables to identify which factors independently predicted disease activity. The abbreviations catalogued in the paper read like a standard inflammatory workup: C-reactive protein, albumin, fibrinogen, hemoglobin, platelet count, neutrophil percentage, fecal calprotectin and fecal occult blood, alongside the Harvey–Bradshaw Index, a bedside clinical score. Four clinical factors survived the selection process and earned a place in the final models, providing the systemic context that imaging alone cannot supply.

The second stream came from computed tomography enterography, a specialized CT technique in which patients drink neutral oral contrast to distend the small bowel, allowing radiologists to inspect the bowel wall for signs of inflammation. Rather than relying on subjective visual reads, the team applied radiomics, the practice of extracting large numbers of quantitative features from medical images, such as texture, intensity and shape descriptors, that are invisible to the human eye. Lesions were outlined using semi-automatic segmentation, and the resulting feature set was pruned with LASSO regression, a statistical technique that shrinks most coefficients to zero and keeps only the most informative predictors. Seven radiomic features made the final cut, each one a compressed mathematical summary of how inflamed tissue looks to a scanner.

The third and most novel stream was the endoscopic image itself. Double-balloon enteroscopy, a technique that uses a balloon-equipped overtube to advance a scope deep into the small bowel, produces images of mucosal lesions that no other modality can capture. The researchers trained ResNet-18, a widely used convolutional neural network architecture built around residual connections that allow very deep networks to train stably, to classify these images and to extract features from their internal layers. From this pipeline they obtained two kinds of output: a binary deep learning classification result, essentially the network’s own verdict on disease activity, and six deep learning-derived features, numerical fingerprints of the endoscopic images that carry information beyond a simple yes-or-no label.

With ingredients in hand, the team assembled seven models spanning the spectrum from single modality to full integration, and the results were unambiguous. Every model achieved an area under the receiver operating characteristic curve, or AUC, exceeding 0.90 in the training cohort, a threshold generally regarded as excellent discrimination. But the multimodal models, designated Combined 2 and Combined 3, which fused clinical data, radiomics and deep learning outputs, delivered the highest performance of all, clearly outperforming both single-modality models and models limited to clinical plus radiomic data. In other words, each additional data stream added genuine predictive value rather than redundant noise.

Crucially, the authors did not rest on AUC alone. They deployed a battery of statistical tests to confirm that the multimodal advantage was real. DeLong tests, the standard method for comparing correlated ROC curves, showed the differences between models were statistically significant. Net reclassification improvement and integrated discrimination improvement, two metrics designed to quantify how much better a new model assigns patients to the correct risk category, both confirmed the superiority of the integrated approach. Calibration analysis checked that predicted probabilities matched observed outcomes, and decision curve analysis evaluated the clinical usefulness of the models across a range of risk thresholds, asking whether acting on the model’s predictions would yield more benefit than treating everyone or no one.

Transparency was another priority. The researchers packaged their best models into nomograms, graphical calculators that let a clinician plot an individual patient’s values on each predictor axis and read off a personalized probability of active disease. To open the black box, they used SHAP, or Shapley Additive Explanations, a technique borrowed from cooperative game theory that assigns each feature a contribution to every individual prediction. The SHAP plots highlighted which clinical, radiomic and deep learning features drove the model’s decisions, offering a measure of interpretability that is increasingly demanded of medical artificial intelligence systems.

The implications extend beyond one hospital in Hefei. Crohn’s disease is a chronic, relapsing condition in which treatment decisions hinge on accurately distinguishing active inflammation from fibrotic scarring or remission, and endoscopic healing has become the gold standard therapeutic target. Yet routine endoscopic assessment of the small bowel requires double-balloon enteroscopy, an invasive, time-consuming and technically demanding procedure, while magnetic resonance and CT enterography are interpreted subjectively and vary between readers. A validated multimodal model could, in principle, let clinicians combine information they already collect, blood tests, a CT scan and endoscopic images, into a quantitative, reproducible activity score, potentially reducing unnecessary procedures and supporting earlier treatment adjustments. The authors are careful to frame the work as complementary to clinical evaluation rather than a replacement for it, and the study’s retrospective, single-center design means prospective validation in larger and more diverse cohorts will be needed before such tools enter practice.

Still, the study offers a compelling demonstration of where medical imaging and artificial intelligence are heading. Rather than betting on a single algorithm or a single modality, it treats the patient as a multimodal data object, with each examination contributing a different slice of the biological truth about inflammation. The fact that models built from routinely acquired clinical, radiological and endoscopic data could all clear the 0.90 AUC bar, and that integration pushed performance higher still, suggests that the raw material for precision medicine in inflammatory bowel disease may already be sitting in hospital archives. As deep learning feature extraction matures and explainability tools like SHAP become standard practice, the gap between research models like this one and bedside decision support continues to narrow, and for patients with small bowel Crohn’s disease, that narrowing could mean fewer invasive exams and better-timed therapy.

Subject of Research: Multimodal machine learning assessment of small bowel Crohn's disease activity using clinical data, CTE radiomics and deep learning on double-balloon enteroscopy images

Article Title: Multimodal evaluation of small bowel crohn’s disease activity integrating clinical data, CTE radiomics, and deep learning on DBE

Article References: Multimodal evaluation of small bowel crohn’s disease activity integrating clinical data, CTE radiomics, and deep learning on DBE. (n.d.). https://doi.org/10.1186/s12880-026-02878-z

Image Credits: AI Generated

DOI: 10.1186/s12880-026-02878-z

Keywords: Crohn's disease, small bowel, computed tomography enterography, radiomics, deep learning, double-balloon enteroscopy, ResNet-18, SHAP, nomogram, inflammatory bowel disease, medical imaging, machine learning