AI Tool Classifies Thyroid Nodule Samples by Bethesda System with High Test Accuracy

Researchers in eastern India have built and publicly released a deep-learning tool, CytoThyro AI, that automatically classifies thyroid fine-needle aspiration cytology (FNAC) images according to the six-category 2023 Bethesda System for Reporting Thyroid Cytopathology. In a prospective study of 175 patients, the Pap-stain-based model ensemble reached a patient-level classification accuracy of 96 percent (26 of 27 patients) on a held-out test set, while a parallel pipeline built on May-Grünwald Giemsa (MGG) stains achieved 89 percent (24 of 27). The work, published open access in Diagnostic Pathology, addresses a practical bottleneck: thyroid FNAC is the standard first investigation for thyroid nodules, but reporting is subject to inter-observer variability, subjectivity and shortages of cytopathologists.

The study team acquired digital microphotographs in both MGG and Papanicolaou stains at 10X and 40X magnifications, generating 3,500 images in total. Bethesda category labels from B1 to B6 were assigned by majority consensus of three blinded cytopathologists, meaning the reference standard was cytological rather than histopathological. Five convolutional neural network architectures were trained with patient-level data partitioning in a 70:15:15 split, so that images from the same patient never appeared in both training and evaluation sets. Final models were chosen using the 2023 Bethesda clinical safety criteria and then combined into a weighted soft-voting ensemble, which was deployed as a publicly accessible web tool at cytothyro.cloud, intended at this stage for research and educational exploration.

Performance details highlight both the promise and the limits of the approach. The Pap-based ensemble achieved a macro F1 score of 0.94 and a weighted F1 of 0.96, with patient-level recall of 1.00 for categories B1, B2, B4, B5 and B6 but only 0.50 for B3. The MGG-based ensemble scored 89 percent accuracy with macro F1 of 0.85 and weighted F1 of 0.88, showing recall of 1.00 for B1 through B4, 0.33 for B5 and 0.80 for B6. The authors note that the category-specific recalls of 1.00 rest on between two and thirteen patients per category and therefore carry wide confidence intervals; the reported 95 percent confidence interval for the Pap pipeline’s accuracy spans 0.82 to 0.99, and for the MGG pipeline 0.72 to 0.96.

The most challenging category was B3, Atypia of Undetermined Significance. At the image level, AUC values for B3 ranged from 0.64 to 0.97 across models, with image-level F1 scores between 0.31 and 0.71. By contrast, the four selected models performed strongly on the clearer categories: image-level AUCs reached 1.00 for B1, ranged from 0.95 to 0.99 for B2, from 0.81 to 1.00 for B4, and from 0.91 to 1.00 for B6. Importantly, from a patient-safety standpoint, neither pipeline produced any B1/B2-to-B5/B6 misclassification at the patient level, meaning benign and malignant-suspicion categories were never confused in the test set.

Several caveats frame the findings. The test set comprised only 27 patients, the reference standard was consensus cytology rather than surgical histopathology, and the study population came from a single region of eastern India. The researchers position CytoThyro AI as a research and educational tool at this stage, not a diagnostic device for patient care. Even so, the study combined prospective image collection, patient-level data splitting, dual staining protocols and a safety-oriented model selection strategy, and the authors position CytoThyro AI as a research and educational tool at this stage.

Subject of Research: Deep learning classification of thyroid fine-needle aspiration cytology images according to the 2023 Bethesda System

Article Title: CytoThyro AI: automated classification of thyroid FNAC images using deep learning based on 2023 Bethesda System for reporting thyroid cytopathology

Article References: Dalal, B. S., Roy, D., Chakrabarti, I., Mondal, S. K., Mazumder, S., Munshi, B. D., Pal, B., Mallick, A., Gon, S., & Mukhopadhyay, K. (2026). CytoThyro AI: automated classification of thyroid FNAC images using deep learning based on 2023 Bethesda System for reporting thyroid cytopathology. Diagnostic Pathology. https://doi.org/10.1186/s13000-026-01862-z

Image Credits: AI Generated

DOI: 10.1186/s13000-026-01862-z

Keywords: thyroid cytology, FNAC, deep learning, Bethesda System 2023, convolutional neural networks, CytoThyro AI, diagnostic pathology, Papanicolaou stain, May-Grünwald Giemsa, AI-assisted diagnosis, inter-observer variability, prospective study