Large Language Models Tested as Clinical Information Sources for Bacteriophage Therapy

Bacteriophage therapy, the therapeutic use of viruses that infect and kill bacteria, has re-emerged as one of the most closely watched strategies in the fight against antimicrobial resistance. Yet the field faces a persistent knowledge problem: phage therapy is highly individualized, deeply technical, and scattered across a literature that spans virology, microbiology, infectious disease medicine, and regulatory science. A new study published in npj Viruses examines whether large language models, the artificial intelligence systems behind modern conversational chatbots, can serve as reliable sources of clinical information on phage therapy, and the findings speak to a broader question about how clinicians should treat AI-generated medical knowledge.

The research, which appears under the title Performance of large language models as a source of clinical information on bacteriophage therapy, was motivated by a practical reality. Physicians considering phage therapy for a patient with a drug-resistant infection often cannot consult a colleague with phage expertise, and formal clinical guidance remains limited. Large language models promise instant, fluent answers to complex medical questions, and surveys suggest that both clinicians and patients increasingly turn to such tools for health information. Whether those answers are accurate, complete, and safe in a niche therapeutic domain like phage therapy had not been systematically assessed, leaving a gap between the enthusiasm for AI-assisted medicine and the evidence needed to support it.

The logic of the evaluation reflects how these models actually work. Large language models are trained on vast corpora of text and generate responses by predicting likely continuations of a prompt rather than by retrieving verified facts from a database. This architecture produces fluent, confident prose regardless of whether the underlying information is correct, a phenomenon often described as hallucination. In a specialized field such as phage therapy, where the training data may be thinner and more heterogeneous than in mainstream medicine, the risk of confident but inaccurate statements is a central concern. The study therefore set out to measure not just whether the models could talk about phage therapy, but whether what they said could be trusted at the bedside.

Phage therapy presents particular challenges for such an assessment. Unlike antibiotics, which are standardized pharmaceutical products, therapeutic phage preparations are typically tailored to the bacterial strain infecting an individual patient. The process involves phage selection, susceptibility testing, formulation, dosing, and monitoring for outcomes that range from bacterial clearance to immune reactions. Clinical evidence includes case reports, small cohort studies, compassionate-use programs, and a limited number of randomized controlled trials, each with different methodological rigor. An information source that conflates experimental findings with established practice, or that presents anecdotal successes as generalizable results, could mislead clinicians in consequential ways.

The evaluation framework used in the study mirrors the standards applied to other emerging medical information tools. Responses generated by the models were assessed for factual accuracy against the primary literature, for completeness in covering the essential elements of a clinical question, for internal consistency, and for the presence of appropriate caveats and safety information. Questions posed to the models spanned the practical spectrum of phage therapy: indications for use, the process of matching phages to bacterial pathogens, dosing and route of administration, known adverse effects, interactions with antibiotics, regulatory status, and the strength of the clinical evidence base. This breadth matters because a model might perform well on general background questions while failing on the specific, operational details that determine whether a therapy is used correctly.

The results highlight a pattern that has emerged across evaluations of AI in medicine. Large language models generally perform well on questions with abundant, well-established answers in the training data. Basic descriptions of what bacteriophages are, how they kill bacteria, and why they are being reconsidered in the era of antimicrobial resistance tend to be accurate and clearly expressed. The models are also effective at summarizing the general rationale for phage therapy and at explaining concepts such as phage specificity and the importance of susceptibility testing. For a clinician seeking orientation in an unfamiliar field, this level of performance can be genuinely useful, providing a readable entry point that would once have required hours of literature searching.

Performance degrades, however, as questions move from general principles to specific clinical detail. The study found that models can produce answers that are partially correct but incomplete, omitting critical caveats such as the experimental status of many phage therapy protocols or the limited availability of approved phage products in most jurisdictions. Some responses blended established facts with outdated or unsupported claims, presenting them with equal confidence. In a domain where treatment decisions depend on precise, current information about phage-bacterium matching and evolving regulatory frameworks, such subtle inaccuracies are not trivial. A response that is ninety percent correct can still be clinically dangerous if the incorrect ten percent concerns dosing, safety, or the evidence supporting a therapeutic claim.

Another dimension of the evaluation concerns how the models communicate uncertainty. Trustworthy medical information sources distinguish clearly between what is proven, what is plausible, and what is speculative. The study indicates that large language models vary considerably in this respect, sometimes providing appropriate disclaimers about the experimental nature of phage therapy and sometimes presenting contested or preliminary findings as settled. This variability is itself informative, because it suggests that clinicians cannot assume a consistent standard of epistemic caution across different questions or different models. The fluency of AI-generated text can mask this inconsistency, making careful verification more important, not less.

The implications extend beyond phage therapy to the broader integration of artificial intelligence into clinical practice. The study’s authors frame their work as a caution against treating chatbots as authoritative references, particularly in specialized and rapidly evolving fields. At the same time, the findings do not support dismissing these tools outright. Used as a starting point for literature exploration, a drafting aid, or a way to formulate better questions for specialists, large language models can add real value. The critical requirement is human oversight: clinicians with domain knowledge must remain in the loop, verifying AI-generated claims against primary sources before any of that information influences patient care. This is the same standard applied to other secondary sources of medical information, and the study argues it should apply with equal force to AI.

The research also points toward what would be needed for large language models to become genuinely reliable clinical resources. Improvements are likely to come from several directions: grounding model responses in curated, up-to-date medical databases rather than relying solely on static training data; developing domain-specific evaluations that test models against expert-validated question sets; and building transparency features that allow users to trace claims back to their sources. For phage therapy specifically, a field whose evidence base is growing quickly as new trials are completed, the ability to incorporate current literature is essential. Until such systems mature, the study’s central message stands: large language models can be informative conversational partners on phage therapy, but their outputs should be regarded as provisional drafts of knowledge, subject to expert review, rather than as substitutes for the primary literature and clinical judgment on which safe patient care ultimately depends.

Subject of Research: Evaluation of large language models as sources of clinical information on bacteriophage therapy

Article Title: Performance of large language models as a source of clinical information on bacteriophage therapy

Article References: Walter, N., Amanatullah, D. F., Debarbieux, L., Doub, J. B., Ferry, T., Groß, J., Międzybrodzki, R., Mirzaei, M. K., Deng, L., Rācenis, K., Suh, G. A., Que, Y.-A., Górski, A., & Rupp, M. (2026). Performance of large language models as a source of clinical information on bacteriophage therapy. npj Viruses, 4(1), Article 41. https://doi.org/10.1038/s44298-026-00224-2

Image Credits: AI Generated

DOI: 10.1038/s44298-026-00224-2

Keywords: bacteriophage therapy, large language models, artificial intelligence, antimicrobial resistance, clinical information, medical AI, npj Viruses, hallucination, infectious diseases, phage selection, clinical decision support, evidence quality