The research team, led by Yuntao Wang and colleagues including Mariluz De Ornelas, Matthew T. Studenski, Elizabeth Bossart, Siamak P. Nejad-Davarani and Yunze Yang, drew its test material from the Radiation Oncology Incident Learning System, or RO-ILS, a national reporting database that collects narrative accounts of errors and near-misses from clinics across the country. Rather than feeding the models raw, unstructured text and hoping for the best, the investigators designed a carefully standardized prompt built on the root cause analysis guidelines of the American Association of Physicists in Medicine. Four state-of-the-art models were put through their paces: Gemini 2.5 Pro, GPT-4o, o3 and Grok 3. Each received the Background and Incident Overview sections from nineteen publicly available RO-ILS cases and was instructed to produce three deliverables for every case: the root causes of the incident, the lessons that should be learned from it, and a set of suggested corrective actions.

What makes this study methodologically interesting is the sheer ambition of its evaluation framework. Assessing the quality of an AI-generated causal analysis is not like checking arithmetic; there is no single correct answer key. The team therefore layered three distinct tiers of assessment on top of one another. At the objective level, they used semantic similarity metrics, computing cosine similarity between model outputs and reference analyses with a Sentence Transformer model, a technique that measures how closely two pieces of text mean the same thing even when they use different words. At a semi-subjective level, they calculated precision, recall and F1-scores, along with an expert-adjudicated positive predictive value, a hallucination rate, and performance criteria covering relevance, comprehensiveness, quality of justification and quality of the proposed solutions. Finally, five board-certified medical physicists provided subjective ratings of reasoning quality and overall performance, bringing genuine clinical judgment to bear on every output.

The headline finding is that the models performed satisfactorily across the board. All four demonstrated comparable baseline capabilities in extracting causal factors from incident narratives, and the analyses they produced were judged relevant and accurate, aligned with what the expert reviewers expected from a competent human analyst. Gemini 2.5 Pro emerged with the highest overall performance score among the four systems, though the differences between models were not uniform across every metric. Statistical testing revealed significant differences among the models in expert-adjudicated positive predictive value, in hallucination rate, and in the subjective ratings assigned by the physicist reviewers, with p-values below 0.05. In plain terms, the models were not interchangeable: some were noticeably more trustworthy than others when their outputs were scrutinized by people who do this work for a living.

That last point matters enormously, because the study did not shy away from the technology’s most notorious weakness. Every model exhibited some degree of hallucination, meaning it fabricated or distorted information not supported by the source material, and the rates ranged from a comparatively modest 11 percent to a troubling 61 percent. For a field where patient safety hangs in the balance, a hallucination rate anywhere near the upper end of that spectrum would be disqualifying for unsupervised use. The authors are careful to frame their results as evidence for assistive rather than autonomous deployment. The vision they describe is not a machine that replaces the safety committee but a machine that drafts the first pass of an analysis, surfaces plausible causal threads, and proposes candidate corrective actions that human experts can then verify, refine and own.

The implications for clinical practice could be substantial. Root cause analysis in radiation oncology typically requires convening multidisciplinary teams of physicists, dosimetrists, therapists, physicians and administrators to reconstruct what happened and why. These sessions are time-consuming, and the narrative reports that feed them are often long, ambiguous and written under stressful circumstances. A language model that can rapidly generate a structured preliminary analysis, organized according to established AAPM guidelines, could shorten the path from incident report to corrective action. It could also help smaller clinics that lack the staffing to conduct exhaustive analyses of every near-miss, potentially raising the baseline of safety surveillance across the entire field. In the aggregate, tools like this could strengthen incident learning systems by making the analytical step faster and more consistent.

The study also offers a template for how medical AI evaluations should be conducted more broadly. Rather than relying on a single metric or a single reviewer, the researchers triangulated across objective similarity measures, quantitative classification metrics and human expert judgment, and they applied statistical significance testing to distinguish genuine performance differences from noise. This layered approach acknowledges an uncomfortable truth about language models: text that looks plausible is not necessarily text that is correct, and similarity to a reference answer does not capture whether a proposed corrective action would actually work in a clinic. By combining expert-adjudicated precision with explicit hallucination measurement, the evaluation gets closer to the question that really matters for patient safety: can this system’s output be trusted, and under what supervision?

Caution remains warranted. Nineteen publicly available cases are a small sample, and publicly reported incidents may differ in complexity and completeness from the internal reports a clinic generates behind its own walls. The models were tested on narrative sections only, without access to the full investigative context that a real RCA team would possess. Hallucination rates, even at the low end, mean that every machine-generated statement about an incident would need verification before it informed any corrective decision. There are also governance questions the study does not resolve: how patient confidentiality would be protected when incident narratives are processed by commercial models, who bears responsibility for an AI-suggested action that proves inadequate, and how such tools would be validated and regulated before entering routine safety workflows.

Still, the direction of travel is clear and, for a field built on the principle of learning from error, quietly exciting. Radiation oncology was among the first medical specialties to confront the reality that complex technology fails in complex ways, and its incident learning infrastructure is among the most mature in healthcare. Injecting language models into that infrastructure, as assistive analysts that draft, summarize and propose while humans verify and decide, could compress the feedback loop between error and improvement from weeks to hours. The PLOS Digital Health study is a proof of concept, not a prescription, but it demonstrates that the reasoning capabilities of current frontier models are already close enough to expert expectations to be worth taking seriously. As hallucination rates fall and evaluation standards mature, the safety committee’s newest member may well be an algorithm, one that never gets tired of reading incident reports and never forgets a lesson once it has been written down.

Subject of Research: Large language model-based root cause analysis of radiation oncology patient safety incidents

Article Title: Augmenting patient safety surveillance in radiation oncology with large language model-based root cause analysis: A proof-of-concept study

Article References: Wang, Y., De Ornelas, M., Studenski, M. T., Bossart, E., Nejad-Davarani, S. P., & Yang, Y. (2026). Augmenting patient safety surveillance in radiation oncology with large language model-based root cause analysis: A proof-of-concept study. PLOS Digital Health, 5(9), e0001740. https://doi.org/10.1371/journal.pdig.0001740

Image Credits: AI Generated

DOI: 10.1371/journal.pdig.0001740

Keywords: large language models, radiation oncology, root cause analysis, patient safety, RO-ILS, incident learning, medical physics, hallucination, AAPM guidelines, artificial intelligence, PLOS Digital Health, quality improvement

Patient safety culture refers to the shared attitudes, beliefs and behaviors that determine how an organization prevents, reports and learns from errors and adverse events. It is the invisible architecture of a hospital: whether a nurse feels able to speak up about a near miss, whether staff believe management will respond constructively to a reported incident, whether error is treated as a personal failing or a systems problem. Decades of research have shown that this culture is one of the strongest predictors of patient outcomes, yet changing it has proven stubbornly difficult. Checklists, posters and mandatory e-learning modules often fail to shift deeply held norms, because they transmit information without engaging the emotional and social machinery that actually drives behavior.

The study took place at Firouzgar Hospital in Tehran in 2023. From a population of 515 nurses, the researchers randomly selected 64 participants using a computer-generated random number sequence and allocated them into two groups of 32. The intervention group took part in six storytelling sessions, each lasting one and a half hours, spread over a three-month period. The sessions were led by a nurse manager, and crucially, the stories were not invented parables or imported case studies. They were drawn from the hospital’s own root cause analyses, the formal investigations conducted after patient safety incidents, and focused on specific events that had actually occurred within those walls.

This grounding in real institutional experience is what distinguishes the approach from conventional safety training. Root cause analysis documents are typically dry, procedural and read by only a handful of quality officers. By converting those documents into narratives, complete with context, human decisions and consequences, the intervention made abstract safety principles concrete and memorable. The control group, meanwhile, received only the routine patient safety education the hospital already provided, creating a clean comparison between standard practice and narrative-based learning.

To measure the effect, the researchers used the Hospital Survey on Patient Safety Culture questionnaire, version 1, a widely validated instrument that assesses multiple dimensions of safety culture, including management support, communication openness, non-punitive responses to error, staffing adequacy and training. Both groups were surveyed at three timepoints: before the intervention began, immediately after the final storytelling session, and one month later. This design allowed the team to track not just whether the intervention worked, but whether its effects persisted after the novelty wore off. Data were analyzed using SPSS software, version 16, with independent t-tests, analysis of variance and Fisher statistical tests used to compare the groups over time.

The results were striking. Patient safety culture scores in the storytelling group rose significantly above those of the control group, with a P value below 0.001, indicating that the probability of the difference arising by chance was less than one in a thousand. The effect size of the intervention was 0.159, which the researchers describe as substantial, since it exceeds the 0.08 threshold conventionally used to distinguish meaningful effects in this type of analysis. In practical terms, nurses who had heard and discussed stories drawn from their own hospital’s incident investigations came to see safety not as a compliance exercise but as a shared professional commitment.

The breakdown across dimensions is particularly revealing. The greatest impact appeared in the training and skills dimension, with an effect size of 0.185, suggesting that narrative learning is especially powerful for building practical competencies. When nurses hear a story about how a communication gap led to a medication error, they do not simply memorize a rule; they rehearse, mentally and emotionally, the situation in which that rule matters. The lowest impact was observed in the staff dimension, with an effect size of 0.049, which makes intuitive sense. Storytelling can change attitudes and skills, but it cannot hire more personnel or fix chronic understaffing, structural problems that require administrative rather than educational solutions.

The study’s design deserves scrutiny, as quasi-experimental studies cannot match the rigor of randomized controlled trials. Participants were randomly allocated to groups, which strengthens the inference, but the intervention was delivered at a single hospital, and the sample of 64 nurses, while adequate for detecting the observed effect, is modest. The researchers also note that the article was published as an early, citable version subject to further editorial refinement. Nevertheless, the three-timepoint measurement, the use of a validated instrument and the random allocation collectively provide what the authors describe as robust evidence that narrative-based learning can complement, and perhaps amplify, routine patient safety education.

The implications reach well beyond one Tehran hospital. Healthcare systems worldwide invest enormous resources in safety training, yet preventable medical errors remain a leading cause of harm. If a low-cost intervention, six ninety-minute sessions led by an existing nurse manager using material the hospital already generates, can produce a measurable and lasting shift in safety culture, the return on investment is extraordinary. The approach also honors a truth that cognitive science has long affirmed: humans encode and recall information far more effectively when it is embedded in narrative than when it is presented as abstract instruction. Stories provide causal structure, emotional salience and social meaning, the very ingredients that checklists lack.

There are also cultural considerations specific to healthcare hierarchies. In many institutions, junior nurses hesitate to question senior colleagues or report incidents involving physicians, fearing blame or career consequences. Storytelling sessions facilitated by a nurse manager create a sanctioned space where error can be discussed openly, where the protagonist of a difficult story is not a villain but a professional navigating a flawed system. By repeatedly modeling this kind of discussion, the intervention may gradually normalize speaking up, which is precisely the behavior that safety culture frameworks identify as foundational. The study, approved by the ethics committee of Iran University of Medical Sciences with informed consent from all participants, offers a template that other institutions could adapt: mine your own incident reports, transform them into honest narratives, and let your staff talk about what really happens when systems fail. The evidence suggests that in the space between the facts of an error and the feelings it provokes, a safer hospital culture can take root.

Subject of Research: The effect of storytelling training based on root cause analyses on patient safety culture among hospital nurses

Article Title: Storytelling for fostering a culture of safety in healthcare: a quasi-experimental study

Article References: Mehraban, M. A., Gharehbaghi-NoveinFar, M., Salehi, T., Haghani, S., & Abbariki, E. (2026). Storytelling for fostering a culture of safety in healthcare: a quasi-experimental study. Health Research Policy and Systems. https://doi.org/10.1186/s12961-026-01508-y

Image Credits: AI Generated

DOI: 10.1186/s12961-026-01508-y

Keywords: patient safety, storytelling, nursing education, safety culture, root cause analysis, quasi-experimental study, HSOPSC, knowledge transfer, hospital quality, narrative learning, Iran University of Medical Sciences, health systems

Researchers assembled a benchmark of 195 published failure analysis cases spanning nine metallurgical categories, from creep and fatigue to stress corrosion cracking, hydrogen embrittlement, and wear. For each case, they extracted a structured narrative describing the component, operating conditions, observations, and laboratory findings, while deliberately withholding the paper’s root cause conclusion. Three large language models were then asked to produce a 200 to 500 word narrative diagnosis, and an automated evaluator compared each diagnosis against the expert-determined ground truth from the original paper.

Without retrieval augmentation, the models performed surprisingly well. Xiaomi’s MiMo-v2.5 correctly identified the root cause in 76.4 percent of cases, DeepSeek-V4-Pro reached 80.0 percent, and DeepSeek-V4-Flash achieved 73.3 percent. When the dense RAG configuration was switched on, every model declined. MiMo-v2.5 fell to 64.1 percent under the filtered dense configuration, a drop of 12.3 percentage points that McNemar’s paired test confirmed as statistically significant (p = .003). The two DeepSeek models showed smaller, non-significant declines of 3.1 and 0.5 percentage points, revealing that the retrieval penalty is strongly model-dependent rather than universal.

The team did not stop at a single retriever. To test whether the degradation was an artifact of dense embedding search, they rebuilt the same 747-document corpus with a lexical BM25 index, the classic keyword-matching algorithm that underlies search engines. BM25 retrieval also failed to beat the no-retrieval baseline, scoring 70.3 percent against 76.4 percent, but it recovered more than half of the dense configuration’s loss. That pattern points the finger at the dense retriever itself: embedding similarity appears to select documents that are semantically similar on the surface yet mechanistically irrelevant to the failure at hand.

Digging into the retrieval logs, the authors identified several plausible culprits. In the default dense setup, duplicate file formats meant 74.4 percent of queries returned duplicate references, leaving only 3.9 unique documents among the requested five. More fundamentally, the dense retriever matched semantic similarity rather than mechanistic relevance, so a case of stress corrosion cracking could retrieve papers about superficially similar corrosion phenomena governed by entirely different physics. The authors frame these as hypotheses consistent with the data, not proven causes.

The error analysis revealed how retrieved context actively corrupts reasoning. In 42 cases, the model diagnosed correctly without retrieval but incorrectly with it, while only 18 cases moved the other way. The researchers describe three recurring patterns: reference anchoring, where the model aligns its conclusion with retrieved papers instead of the input evidence; mechanism contamination, where retrieved documents introduce failure mechanisms absent from the case; and confidence suppression. Common misdiagnoses included confusing fatigue with creep, mistaking stress corrosion cracking for corrosion fatigue, and blaming environmental factors when the true cause was a manufacturing defect.

Performance also varied by failure category, though the authors caution that small sample sizes make these patterns indicative rather than conclusive. Filtered dense RAG was most damaging for brittle fracture, where accuracy collapsed by 29.4 percentage points, followed by stress corrosion cracking and ductile overload, categories that often involve multi-factorial mechanisms sensitive to partially relevant context. Wear and fretting cases achieved the highest baseline accuracy at 91.7 percent. Notably, the retrieval corpus overlapped thematically with the ground-truth literature, which in principle could have inflated RAG’s apparent value, making the observed penalty all the more notable.

The evaluation methodology itself was carefully validated. Three independent failure analysis experts blind to the retrieval configuration reviewed stratified samples of both non-RAG and RAG outputs, achieving substantial agreement with one another. The automated evaluator agreed with expert consensus in 70.0 percent of non-RAG cases and 84.0 percent of RAG cases, and critically, it never rated a case correct when the experts rated it wrong. Using Gwet’s AC1, chosen over Cohen’s kappa for its stability with imbalanced categories, the team showed the reported accuracy differences are unlikely to be artifacts of evaluator unreliability.

The practical implications are pointed. Current LLMs appear viable as screening tools for text-based root cause analysis, with the best configuration correctly ruling out wrong mechanisms in 93.3 percent of cases, and the entire evaluation pipeline cost roughly US$0.43 in API fees. But the default assumption that bolting a literature database onto a model improves its analysis is not supported in this domain. The authors recommend deduplicating corpora before indexing, building mechanism-aware indexes rather than purely semantic ones, extracting mechanism-specific keywords for queries, and activating retrieval only when model confidence is low.

Limitations remain. Only one model family was tested across all four retrieval configurations, the pipeline processed text but not the micrographs and fractographs metallurgists routinely rely on, and some benchmark cases may have been encountered during pretraining, though memorization would affect both configurations equally and cannot explain the retrieval penalty. The conclusions apply to the specific retrieval setups evaluated, not to RAG in general. Still, the message lands with unusual force for the AI engineering community: in domains where models already carry deep internalized knowledge, indiscriminate retrieval does not add information, it adds noise, and the most expensive context can be the wrong context.

Subject of Research: Retrieval-augmented generation effects on large language model performance in metallurgical failure root cause analysis

Article Title: When RAG hurts: Retrieval-augmented generation degrades performance in metallurgical root cause analysis

Article References: Fauzi, A. R., Pratama, S. N., Purqon, A., Ramadhan, Y. Z., & Ardiansyah, M. (2026). When RAG hurts: Retrieval-augmented generation degrades performance in metallurgical root cause analysis. Results in Engineering, 32, Article 113136. https://doi.org/10.1016/j.rineng.2026.113136

Image Credits: AI Generated

DOI: 10.1016/j.rineng.2026.113136

Keywords: retrieval-augmented generation, large language models, metallurgical failure analysis, root cause analysis, RAG degradation, dense retrieval, BM25, materials science, artificial intelligence, failure mechanisms, DeepSeek, MiMo-v2.5