The research team, led by sonographers and clinicians from several Chinese institutions, tested six mainstream LLMs, including DeepSeek-V3, ChatGPT-4o-Latest, GPT-3.5-Turbo, Doubao-pro-128k, GLM-4-plus, and Qwen-plus, on their ability to assign C-TIRADS grades to thyroid nodules using only written ultrasound reports. C-TIRADS, the Chinese Thyroid Imaging Reporting and Data System introduced in 2020, stratifies nodules into risk categories based on features such as margins, echogenicity, shape, and microcalcifications. Categories 4A through 5 carry escalating malignancy risks, from 2-10 percent at 4A to over 90 percent at Category 5, with papillary thyroid carcinoma (PTC) as the dominant malignant subtype.

To give the models a fighting chance at consistency, the researchers built a FastGPT knowledge base using Retrieval-Augmented Generation (RAG) technology, populated with 11,489 de-identified thyroid ultrasound records collected at the same hospital between 2022 and 2023. This corpus was vectorized using a Chinese-language embedding model and parsed with GLM-4 to produce structured reference material. When a new report arrives, the RAG system retrieves the most relevant historical cases and guideline fragments, injecting them as context before the LLM renders its verdict. The entire pipeline ran inside Docker containers on the hospital’s internal intranet, with only fully de-identified text ever leaving the network for closed-source commercial models.

The test bed was a set of 228 thyroid ultrasound reports from December 2024, stripped of all physician-assigned grading labels to prevent the models from simply copying embedded conclusions. Three senior sonographers with more than 15 years of experience independently graded the same cleaned text, producing a consensus benchmark. Crucially, the authors emphasize that this text-based consensus is not a pathological gold standard; it reflects how experienced humans interpret the same words the models see, not ground truth about whether a nodule is actually cancerous.

Among the standalone models, DeepSeek-V3 led the pack with an overall grading concordance of 81.14 percent against the expert consensus. But the real story emerged when the knowledge base was switched on. Overall concordance across all six models rose significantly from an average of 70.39 percent to 79.75 percent (P < 0.001). Doubao-pro-128k, ByteDance’s Chinese-language model, surged from 77.63 percent to 89.04 percent, the highest of all twelve pipelines tested, with a linear-weighted Kappa exceeding 0.8 against expert grading after Bonferroni correction. GLM-4-plus jumped by nearly 19 percentage points, and ChatGPT-4o-Latest improved by more than 11 points.

Speed was equally striking. Human sonographers averaged 19.25 seconds per case; the twelve AI pipelines averaged just 1.30 seconds, roughly fifteen times faster. GPT-3.5-Turbo was the sprint champion at 0.28 seconds per case, though knowledge-base integration slowed models somewhat, with Doubao-pro-128k taking 1.70 seconds when augmented. The trade-off between retrieval overhead and accuracy gains appears favorable for most models tested.

Yet the improvement was not uniform across risk tiers. Concordance rose significantly for C-TIRADS Categories 1 and 3, the low- and intermediate-risk nodules that dominate daily practice, and increased numerically for Category 2. But for the borderline Category 4A nodules, the very lesions where benign and malignant features overlap and sonographers disagree most, concordance actually declined numerically from 63.00 percent to 56.33 percent, though the drop was not statistically significant. The authors suggest that the knowledge base, filled with real-world 4A cases sharing overlapping features, may paradoxically make these ambiguous nodules harder to classify definitively rather than easier.

To probe whether the models’ risk tiers actually track biology rather than merely mimic human text, the team conducted an exploratory analysis on a small subgroup of 62 nodules with definitive pathology from fine-needle aspiration or surgery, comprising 12 papillary thyroid carcinomas and 50 benign lesions. Knowledge-enhanced Doubao-pro-128k produced a statistically significant ascending malignant risk gradient across C-TIRADS tiers (Mantel-Haenszel trend chi-square P < 0.001) and achieved an AUC of 0.783 (95% CI: 0.635-0.932) for distinguishing benign from malignant lesions, the highest among all pipelines. At the C-TIRADS ≥ 4A decision threshold, it also posted the highest Youden index at 0.277, with sensitivity of 0.917.

By contrast, knowledge-enhanced DeepSeek-V3 produced a negative Youden index of −0.147, indicating worse-than-chance alignment with pathology at that cutoff, despite its strong text-level concordance. The authors are careful to note that these divergent results underscore a fundamental distinction: matching human textual reasoning is not the same as discriminating actual malignancy. The pathological subgroup was small and imbalanced, so all such findings remain exploratory and require validation in larger multi-center cohorts before any clinical thresholds can be drawn.

The study’s authors are equally candid about its limits. The test set contained only 11 Category 4B nodules, zero Category 4C, and a single Category 5 case, reflecting the reality that Chinese tertiary hospitals routinely biopsy anything graded 4A or higher, leaving few untreated high-risk nodules in retrospective datasets. The work was single-center, retrospective, and text-only, and cannot speak to downstream outcomes such as biopsy rates or patient prognosis. Still, the authors argue that the pipeline’s real value lies in standardizing the messy free-text reports that dominate clinical ultrasound documentation, potentially narrowing the grading gaps between junior and senior sonographers and enabling batch quality control at negligible labor cost. They stress that any AI-generated 4A finding must remain subject to mandatory review by attending sonographers, and that model outputs should serve only as supplementary references, never as independent grounds for biopsy or surgical decisions.

Subject of Research: Use of large language models with a retrieval-augmented knowledge base to improve C-TIRADS risk grading of thyroid nodules on ultrasound reports

Article Title: Application of LLM combined with knowledge base in improving C-TIRADS grading of thyroid nodules

Article References: Qu, Q., Bi, H., Yu, X., Zhou, R., Liu, Z., & Hao, T. (2026). Application of LLM combined with knowledge base in improving C-TIRADS grading of thyroid nodules. Frontiers in Endocrinology, 17, Article 1888772. https://doi.org/10.3389/fendo.2026.1888772

Image Credits: AI Generated

DOI: 10.3389/fendo.2026.1888772

Keywords: thyroid nodules, C-TIRADS, large language models, retrieval-augmented generation, FastGPT, ultrasound, papillary thyroid carcinoma, artificial intelligence, DeepSeek-V3, Doubao-pro-128k, diagnostic concordance, medical imaging