AI Chatbots Answer Gastric Cancer Questions but Struggle to Spot It in Endoscopy Images

Large language models including ChatGPT, DeepSeek, Gemini and Grok can answer written clinical questions about early gastric cancer with reasonable quality, but they perform poorly and inconsistently when asked to recognise the disease in static endoscopic images, according to a study published in BMC Gastroenterology.

The research team, led by authors including Changda Lei and Rui Li, built two complementary benchmarks. For the text evaluation, they developed 40 clinical questions on early gastric cancer together with 50 single-choice questions based on clinical guidelines. Physicians and patients then rated the answers generated by different models on four dimensions: accuracy, relevance, completeness and comprehension, and the responses were also assessed for readability. For the image evaluation, the group assembled 482 white-light endoscopy images and 478 narrow-band imaging images from four hospitals in China, testing whether the models could identify early gastric cancer in still photographs of the stomach lining.

On the written questions, ChatGPT-4o achieved the highest scores for accuracy (86), relevance (88) and comprehension (88), each statistically significant at P < 0.05, while Grok-3 led on completeness with a score of 90, also at P < 0.05. In the single-choice format, DeepSeek-V3.2 recorded the highest accuracy at 88%, although the difference between models was not statistically significant (P = 0.268). These results indicate that, for guideline-based clinical questions, the leading chatbots already produce answers that human raters judge to be largely accurate, relevant and understandable to both clinicians and patients.

The picture changed sharply for image interpretation. In white-light endoscopy images, the reported sensitivity for pathological type recognition was 65.08% for ChatGPT-5.2, 99.60% for Gemini-3.0 pro and 81.75% for Grok-4, but specificity told a very different story: 47.92% for ChatGPT-5.2, 0% for Gemini-3.0 pro and 29.17% for Grok-4. For the second image set, the reported sensitivities were 57.54%, 99.60% and 92.86% respectively, with specificities of 60.42%, 0% and 2.08%. A specificity of 0% means a model flagged essentially every image as positive, which inflates sensitivity while producing large numbers of false positives.

The authors conclude that large language models show potential in addressing clinical questions related to early gastric cancer, but their ability to recognise the disease in static multimodal endoscopy images is limited and their effectiveness is inconsistent.

The study has clear scope limitations that readers should keep in mind. The image dataset came from four hospitals in China, so performance may differ on images from other centres, equipment or patient populations. The evaluation used static images rather than the live, video-based examination that occurs in real endoscopy suites, and the text benchmark was built around guideline-derived questions rather than open-ended clinical dialogue. The article was also shared early as a peer-reviewed, accepted manuscript that remains citable but subject to further edits before the final version of record.

Even so, the contrast between the two halves of the evaluation is instructive.

Their use as diagnostic readers of endoscopic images, however, is not supported by these findings, and the authors’ own conclusion frames that capability as limited and inconsistent rather than emerging.

Subject of Research: Performance of large language models on early gastric cancer clinical questions and endoscopic image recognition

Article Title: Comparison of large language models in answering early gastric cancer clinical problems and recognizing multimodal endoscopic images

Article References: Lei, C., Dai, C., Chen, L., Xu, S., Guo, T., Kan, X., Ouyang, Y., Mei, Y., Wang, C., Wang, B., Hong, K., Li, J., Guo, Y., & Li, R. (2026). Comparison of large language models in answering early gastric cancer clinical problems and recognizing multimodal endoscopic images. BMC Gastroenterology. https://doi.org/10.1186/s12876-026-05474-1

Image Credits: AI Generated

DOI: 10.1186/s12876-026-05474-1

Keywords: large language models, early gastric cancer, endoscopy, ChatGPT, DeepSeek, Gemini, Grok, white-light endoscopy, narrow-band imaging, artificial intelligence, gastroenterology, medical AI evaluation