Six AI models read thousands of cancer narratives and mostly disagree on what they feel

When people write about cancer online, the words they choose carry fear, sadness, uncertainty and, less often, hope. A new study in Discover Analytics set out to read those words at scale, applying six different natural language processing tools to a corpus of publicly accessible cancer-related texts collected from more than 200 online sources, including blogs, discussion forums, patient stories, support communities and selected informational pages. The result is less a single verdict on how cancer patients feel than a detailed portrait of how differently machines read the same emotional language.

The research team, led by Md. Murad Hossain and Mohammad Abdul Halim, with Yasir Arfat and Md Abdullah Al Rahat as co-authors, built their corpus from English-language texts that contained cancer-related experiential or discussion content, excluding duplicates, navigation-only pages, advertisements and irrelevant material. No medical records or clinician-restricted documents were used at any stage. The texts were gathered with automated scripts and standard web extraction tools, then cleaned through duplicate removal, HTML stripping, normalization, tokenization and filtering of low-quality content. The authors stress that the dataset represents public online discourse rather than a clinically curated or demographically representative patient sample.

Onto this corpus the team unleashed six analytical approaches: the lexicon-based tools NRCLex and VADER, the recurrent neural networks LSTM and BiLSTM, and the transformer models DistilBERT and RoBERTa. Each tool was asked to classify sentiment as positive, negative or neutral, both across self-reported cancer stages and across treatment-related categories such as chemotherapy, radiation, surgery, immunotherapy, palliative care, diagnosis and targeted therapy. Stage and treatment labels were assigned on the basis of explicit mentions within the narratives, not clinical records, making them mention-based analytical groupings rather than verified patient trajectories.

The most striking methodological finding concerns how much the tools diverged. NRCLex, VADER and DistilBERT all found negative sentiment dominating at every cancer stage, with the highest raw negative counts appearing in stage IV narratives. RoBERTa, by contrast, labelled neutral sentiment as the dominant category at all four stages, classifying 69 of 97 stage I sentences as neutral, for example, and 102 stage IV sentences as neutral. The authors attribute this to the model’s decision threshold rather than to the true emotional content of the texts, and note that without gold-standard labels it is impossible to adjudicate between the divergent readings the four tools assign to the same text.

Statistical testing sharpened the picture. At the stage level, chi-square tests of independence found no statistically significant association between cancer stage and sentiment classification for any of the four main models, with Cramér’s V values of roughly 0.07 to 0.08 and unadjusted p-values all above 0.05. The apparent rise in negative sentiment at stage IV was consistent with a greater volume of stage IV text in the corpus rather than a genuine shift in polarity. At the treatment level, however, the associations were statistically significant for all four models, with sample sizes between 22,672 and 26,558 scored mentions per model and Cramér’s V effect sizes between 0.12 and 0.25, indicating small-to-moderate effects. Diagnosis-related narratives contained the largest volume of scored sentiment across every model, marking diagnosis as the most emotionally dense context in the corpus.

The treatment-level patterns themselves were revealing. NRCLex counted 5,787 negative versus 3,591 positive sentiment instances for diagnosis, and chemotherapy, radiation and surgery were all dominated by negative sentiment, with radiation showing 2,533 negative against 728 positive counts. Palliative care stood out with a strong positive skew of 1,017 positive versus 215 negative instances, which the authors link to its comfort-focused role. Immunotherapy and targeted therapy displayed more mixed profiles, suggesting a combination of optimism, uncertainty and caution in discussions of newer treatments. VADER and DistilBERT produced broadly convergent patterns, with diagnosis again carrying the heaviest emotional weight.

Emotion classification added a further layer of divergence. NRCLex, which scores word-level emotional cues, found fear overwhelmingly dominant, with 144,574 occurrences, followed by trust at 45,085, while joy appeared only 73 times. The transformer-based classifier painted an almost opposite picture: caring emerged as the most frequent emotion at 143,089 instances, followed by realization at 64,306 and curiosity at 35,015, with acute negative emotions rare. The authors treat this contrast as an illustration of how lexicon-based and transformer-based tools disagree on the same corpus, rather than resolving which tool is correct. A T5-based classifier applied to treatment-specific emotion found sadness most prevalent, concentrated in diagnosis-related text at 2,979 instances, with fear also prominent around diagnosis and surgery.

The recurrent models were trained and evaluated on 65,103 sentence-level instances using labels derived automatically from VADER scores rather than human annotation, in a 70:20:10 split. Both LSTM and BiLSTM achieved 76 percent accuracy, but this figure represents agreement with the VADER-derived proxy labels, not with independent human judgement. The more informative metrics were the macro-averaged F1 scores of 0.46 for LSTM and 0.49 for BiLSTM, which expose how strongly both models favoured the majority neutral class, containing 48,286 instances, over the minority negative and positive classes. Because sentences from the same source document could appear in both training and test partitions, the authors acknowledge that these performance estimates may be optimistic.

The study’s ethical framing is unusually explicit for computational text analysis. The authors analysed only aggregate, de-identified data, made no attempt to re-identify or geolocate authors, restricted collection to sources whose terms of service permitted automated access, and note that public availability of a post does not constitute informed consent for research use. They acknowledge that these measures fall short of individual informed consent and that no formal institutional review board exemption letter was documented, positioning the work instead under a commonly applied exemption rationale for secondary analysis of existing public data.

Above all, the authors are careful about what the study does and does not show. The findings are presented as corpus-level discourse patterns consistent with, rather than novel additions to, the existing psycho-oncology literature on emotional trajectories across the cancer journey, and explicitly not as clinical discoveries or a validated diagnostic tool. The principal contribution, they argue, is methodological: a systematic comparison of how lexicon-based, recurrent and transformer-based approaches classify the same emotionally rich texts. Future work, they write, should prioritise manually annotated gold-standard labels, document-level train-test separation, evaluation of domain-specific models such as BioClinicalBERT, and closer integration with psycho-oncology theory through designs developed in partnership with clinicians and patients.

Subject of Research: Sentiment and emotion analysis of publicly posted online cancer narratives using multiple natural language processing models

Article Title: Detecting emotional distress in cancer patients through sentiment analysis

Article References: Hossain, M. M., Arfat, Y., Al Rahat, M. A., & Halim, M. A. (2026). Detecting emotional distress in cancer patients through sentiment analysis. Discover Analytics, 4(1), Article 38. https://doi.org/10.1007/s44257-026-00087-0

Image Credits: AI Generated

DOI: 10.1007/s44257-026-00087-0

Keywords: sentiment analysis, cancer narratives, natural language processing, NRCLex, VADER, RoBERTa, DistilBERT, LSTM, BiLSTM, psycho-oncology, emotional distress, text mining