LDA pathology features lead thyroid cancer recurrence model

A machine-learning model combining structured medical-record variables with features derived from pathology reports showed moderate performance for predicting recurrence of differentiated thyroid cancer in a retrospective study of 310 patients. The best-performing approach was an XGBoost classifier augmented with latent Dirichlet allocation, or LDA, features from surgical and cytopathology reports. After hyperparameter tuning, it achieved an F1 score of 0.631 and an area under the receiver operating characteristic curve of 0.741 on a held-out test set. The results support the feasibility of extracting predictive information from free-text reports, but the model’s poor calibration, small number of recurrent cases and lack of external validation limit its clinical applicability.

The analysis used records from adult thyroid cancer patients treated at the Icahn School of Medicine at Mount Sinai between January 1, 2010 and December 31, 2022. Investigators initially identified 2,397 patients through International Classification of Diseases codes. The analysis included 310 patients whose final surgical pathology confirmed differentiated thyroid cancer and for whom demographic, laboratory, cytopathology and surgical pathology data were available. Thirty-three patients, or 10.6%, experienced recurrence, while 277 had no evidence of disease recurrence. Recurrence was defined as biopsy-proven differentiated thyroid cancer identified after the index surgery; patients with metastatic disease at the time of surgery were not classified as recurrent cases.

Researchers tested three methods for converting unstructured pathology text into model inputs. Term Frequency–Inverse Document Frequency, or TF-IDF, measures the importance of individual words by combining their frequency within a document with their prevalence across a collection. LDA was used as an unsupervised topic model to identify clusters of related words, then the resulting topic probabilities were supplied to the prediction algorithms as numerical features. The third method used Meta’s Llama 3.1 8B large language model in a zero-shot classification task. Surgical and cytopathology reports were combined and entered into a prompt requesting a binary recurrence label, with the model temperature set to zero for consistency. The standalone large language model performed poorly, with an F1 score of 0.393 and accuracy of 0.542, although its output was retained as an auxiliary feature in some configurations.

Five learning algorithms—decision tree, gradient boosting, logistic regression, random forest and XGBoost—were evaluated under six combinations of structured and NLP-derived inputs, producing 30 model evaluations. Structured electronic medical-record variables included age at surgery, sex, race, insurance type, preoperative body mass index, family history and postoperative thyroglobulin level. All models were assessed using stratified five-fold cross-validation, with macro F1 score as the primary criterion for selecting the best model. The selected XGBoost model was tuned with a grid search and then evaluated once on a separate test set containing 93 patients, including 10 with recurrence. The 95% confidence intervals were estimated through 1,000 iterations of stratified bootstrap resampling.

Among models using structured records alone, gradient boosting had the highest training F1 score, 0.521, but a ROC-AUC of 0.457. Adding LDA features produced the strongest results. Before tuning, XGBoost with electronic-record and LDA inputs reached a cross-validated F1 score of 0.591 and ROC-AUC of 0.744. Hyperparameter tuning increased the training F1 score to 0.647 and ROC-AUC to 0.753. On the held-out test set, the final model achieved an F1 score of 0.631, with a 95% confidence interval of 0.509–0.758, and a ROC-AUC of 0.741, with a 95% confidence interval of 0.572–0.892. Its test-set accuracy was 0.817, precision was 0.616, recall was 0.677, and precision-recall AUC was 0.317.

The model’s estimated probabilities were not well calibrated, an important qualification for a clinical prediction system. Its Brier score was 0.141, higher than the null-model score of 0.096, indicating that its predicted probabilities were less accurate than the prevalence-based benchmark in this test set. A separate logistic regression model using postoperative thyroglobulin alone reached a cross-validated ROC-AUC of 0.643, compared with 0.744 for the combined electronic-record and LDA XGBoost model during training. That comparison suggests the added features may provide incremental discrimination beyond thyroglobulin alone in this cohort. The authors nevertheless cautioned that the combined model should not be interpreted as clinically applicable, and calibration may differ in populations with different baseline recurrence rates.

Two approaches to explaining the model identified lymph-node-related pathology topics as the dominant signals. Gain-based feature importance and SHAP analyses both placed a surgical pathology LDA topic concerning cervical and lateral neck lymph node dissection at the top of the ranking. Other prominent features included cytopathology topics related to malignancy, several surgical pathology topics describing lymph-node dissection and the extent of cancer involvement, Asian race, female sex, postoperative thyroglobulin, and topics concerning general pathology findings and fine-needle aspiration preparation. Seven of the 10 highest-ranked features were LDA-derived. The dominant surgical topic contained words including “lymph,” “node,” “level,” “dissection,” “neck,” “carcinoma,” “metastatic” and “tumor.”

The importance of this topic does not establish that lymph-node dissection independently causes recurrence. The authors noted that it may instead serve as a proxy for underlying nodal disease, the extent of surgery and broader disease burden, which can be associated with recurrence. The clinically coherent pattern emerged through an unsupervised process that did not require researchers to define and manually extract each predictive phrase in advance. An endocrine surgeon reviewed the topics and assigned descriptive names. However, the study did not compare the NLP pipeline with a model trained on manually abstracted surgical pathology data, so the independent contribution of automated extraction could not be isolated.

Several design features limit the study’s generalizability. The cohort came from a single tertiary referral institution, where some laboratory results and cytopathology reports originated outside the health system and were not captured as extractable electronic records. Excluding patients with incomplete data may therefore have introduced selection bias. The study included only 33 recurrent cases, creating substantial class imbalance that can make model performance unstable. The researchers did not establish a minimum follow-up duration or obtain individual follow-up data, preventing time-to-event analysis and making it possible for patients with shorter observation to be labeled non-recurrent despite a later recurrence. Established recurrence predictors such as AJCC/TNM stage and American Thyroid Association risk category were not included because they were not readily available as structured variables. Only one large language model was tested, without prompt refinement or domain-specific fine-tuning.

Within those boundaries, the study shows a practical route for turning narrative pathology reports into structured inputs for recurrence modeling. LDA captured a dominant neck-dissection theme that the authors considered clinically meaningful, whereas TF-IDF and zero-shot large language model features did not appear in the best-performing configuration. The findings therefore favor an interpretable topic model over those two alternative representations for this dataset and task, rather than showing that word-frequency methods or large language models are ineffective in clinical NLP generally. Multi-institutional cohorts with more recurrent cases, standardized follow-up, complete staging variables, external validation and direct comparisons with manual abstraction will be needed to determine whether the observed performance persists. Until then, the work provides evidence of modeling feasibility, not a validated method for estimating an individual patient’s absolute recurrence risk.

Subject of Research: Natural language processing of pathology reports for differentiated thyroid cancer recurrence prediction

Article Title: Incorporating natural language processing-derived features from pathology reports into machine learning models to predict differentiated thyroid cancer recurrence

Article References: Lee, D. T., Yen, S., Deng, G., Elkersh, Y., Thomas, A. M., Govindarajulu, U., Marin, M. L., Nadkarni, G. N., & Matteson, D. S. (2026). Incorporating natural language processing-derived features from pathology reports into machine learning models to predict differentiated thyroid cancer recurrence. Frontiers in Endocrinology, 17, Article 1939077. https://doi.org/10.3389/fendo.2026.1939077

Image Credits: AI Generated

DOI: 10.3389/fendo.2026.1939077

Keywords: differentiated thyroid cancer, cancer recurrence, pathology reports, natural language processing, latent Dirichlet allocation, machine learning, XGBoost, electronic health records, clinical NLP, thyroglobulin, Incorporating, natural