The researchers set out to map where, in the machinery of modern healthcare delivery, AI-related harms actually occur. Rather than treating AI as a single black box that a doctor either accepts or rejects, they categorized the cases according to the specific clinical tasks physicians perform: diagnosing conditions, ordering and interpreting tests, planning treatment, triaging patients, documenting and communicating information, and coordinating care across institutions. The distribution was revealing. Harms clustered not only around diagnostic tools — the category that dominates public imagination and much of the regulatory conversation — but also around administrative and coverage systems, including AI used by insurers to deny or curate care, and screening tools deployed by state agencies to flag families for investigation.

That breadth matters because it exposes a structural blind spot in both technology design and legal doctrine. Much of the existing scholarship on AI liability in medicine has centered on a deceptively simple question: if an AI tool outperforms a doctor, does reliance on it constitute negligence, or does deviation from it? Prior work, including widely cited analyses in JAMA and law reviews, has debated whether physicians could face liability for ignoring a competent algorithm or for deferring to a flawed one. Mansi and Riedl argue that this physician-centered frame is too narrow to describe what actually happens when patients are harmed. An insurance company’s algorithm may terminate home-care coverage; a state screening tool may incorrectly flag a parent for child-welfare investigation; a triage system may misdirect emergency care. In each scenario, the patient’s injury emerges from decisions distributed across an organization, with the physician often having little visibility into — or control over — how the AI was configured, trained, deployed or applied.

The legal cases the researchers analyzed come from a meticulous triangulation of three public databases: the Database of AI Litigation maintained at George Washington University Law School, the Health Litigation Tracker, and the AI, Algorithmic and Automation Incidents and Controversies repository, a community-maintained archive of documented harms. As of February 2025, all three resources are freely accessible online, and the researchers referenced individual cases through unique identifiers, centering documented harms rather than the specific systems involved. The case-study approach allowed the authors to reconstruct, case by case, how harm materialized, which stakeholders were positioned between the patient and the algorithm, and what barriers the patient encountered in seeking recourse.

Those barriers, the study finds, are both legal and technological, and they reinforce one another. On the legal side, liability structures remain tethered to a model of medical malpractice that presumes a human decision-maker with enough information and discretion to have acted differently. When an algorithm’s output is embedded in a payer’s coverage workflow or a state agency’s screening pipeline, a patient’s lawyer must identify which entity’s conduct was negligent, prove causation, and often overcome trade-secret protections that keep the model’s logic opaque. On the technological side, the tools themselves are rarely designed for contestability. Explanations, where they exist, are built for clinicians — and even for clinicians, a substantial body of human-computer interaction research shows that explainability features are frequently underused, misunderstood or shaped more by regulatory checkbox than by genuine decision support. They are almost never built for the people who most need them when things go wrong: patients and their advocates.

Here the study makes its most provocative move. Mansi and Riedl argue that lawyers working for patients should be recognized as legitimate users of AI systems — and, in a sense, as intermediaries whose needs deserve to shape the design of medical AI from the ground up. If an explainability interface cannot help a patient’s legal team understand which stakeholder made which consequential decision, what data the model relied on, and where in the pipeline the error propagated, then the tool has effectively insulated everyone but the patient from scrutiny. The authors connect this to a growing research program on legally-informed explainable AI, which aims to produce explanations aligned with the evidentiary demands of legal proceedings rather than with the aesthetic of transparency that currently dominates industry practice.

The researchers also draw a pointed lesson from documented incidents involving coverage algorithms. Reporting on how Medicare Advantage plans have used AI tools to cut off care for seniors, alongside legal scholarship on regulating healthcare coverage algorithms, illustrates the pattern the study identifies: harms arise at the intersection of commercial incentives, algorithmic automation and fragmented oversight. A patient denied post-acute care by an algorithmic coverage tool may appeal through administrative channels, sue, or simply suffer silently. Each path demands different evidence, different expertise and different institutional access — resources that are unevenly distributed among the very populations most likely to be harmed.

What can be done? The authors propose reform on two fronts. First, liability structures should evolve to reflect the distributed reality of AI-mediated care. They point to legal scholarship exploring theories such as the common enterprise theory of liability, which could allow responsibility to be shared across the ecosystem of actors — developers, payers, facilities, regulators — that jointly shape how a tool affects patients. Distributed governance frameworks for medical AI, and regulatory thinking that treats software as a medical device within a whole system rather than as a standalone artifact, offer templates for how accountability could be allocated more realistically. Second, AI tools for healthcare should be designed for contestability by default: systems should log which stakeholders shaped their outputs, generate explanations usable in legal contexts, and support advocates — including patients’ lawyers — in reconstructing what happened when harm occurs. Contestable AI, an emerging design philosophy in human-computer interaction, offers concrete principles: make decisions reversible, make the grounds of decisions accessible, and make the pathway to human review visible to affected people.

The stakes of getting this wrong extend beyond individual lawsuits. If patients cannot seek recourse, there is no corrective feedback signal pushing developers and deployers to fix flawed systems; courts become the only place where the accumulated evidence of algorithmic failure can surface, and only for those few patients wealthy or persistent enough to get there. Conversely, if accountability is designed into the system — technologically through auditable, contestable tools, and legally through liability frameworks that track the actual distribution of decision-making power — then recourse becomes possible without requiring every patient to become a litigant. The study’s authors, whose work was supported in part by the National Science Foundation, frame this as a shift from physician-centered to patient-centered accountability, a reorientation that demands new collaboration between lawyers, technologists, clinicians and regulators.

For a healthcare system rapidly absorbing AI into triage, diagnosis, documentation and coverage decisions, the message of the litigation record is uncomfortable but clear: the technology is arriving faster than the accountability structures needed to govern it, and the people bearing the consequences are the ones the current framework serves least. Mansi and Riedl’s analysis of 31 cases turns courtroom silence into a design specification — a map of where AI harms happen in healthcare delivery, and a blueprint for tools and laws that could finally let patients fight back.

Subject of Research: Analysis of US litigation involving medical AI tools and patient-centered accountability for AI-related harms in healthcare delivery.

Article Title: Implications of current litigation on the design of AI tools for healthcare delivery and related legal frameworks

Article References: Mansi, G., & Riedl, M. (2026). Implications of current litigation on the design of AI tools for healthcare delivery and related legal frameworks. Nature Health. https://doi.org/10.1038/s44360-026-00186-y

Image Credits: AI Generated

DOI: 10.1038/s44360-026-00186-y

Keywords: medical AI, healthcare litigation, AI liability, patient recourse, explainable AI, contestable AI, healthcare delivery, algorithmic harms, coverage algorithms, legal frameworks, patient-centered accountability, Nature Health

The research, which appears under the title Performance of large language models as a source of clinical information on bacteriophage therapy, was motivated by a practical reality. Physicians considering phage therapy for a patient with a drug-resistant infection often cannot consult a colleague with phage expertise, and formal clinical guidance remains limited. Large language models promise instant, fluent answers to complex medical questions, and surveys suggest that both clinicians and patients increasingly turn to such tools for health information. Whether those answers are accurate, complete, and safe in a niche therapeutic domain like phage therapy had not been systematically assessed, leaving a gap between the enthusiasm for AI-assisted medicine and the evidence needed to support it.

The logic of the evaluation reflects how these models actually work. Large language models are trained on vast corpora of text and generate responses by predicting likely continuations of a prompt rather than by retrieving verified facts from a database. This architecture produces fluent, confident prose regardless of whether the underlying information is correct, a phenomenon often described as hallucination. In a specialized field such as phage therapy, where the training data may be thinner and more heterogeneous than in mainstream medicine, the risk of confident but inaccurate statements is a central concern. The study therefore set out to measure not just whether the models could talk about phage therapy, but whether what they said could be trusted at the bedside.

Phage therapy presents particular challenges for such an assessment. Unlike antibiotics, which are standardized pharmaceutical products, therapeutic phage preparations are typically tailored to the bacterial strain infecting an individual patient. The process involves phage selection, susceptibility testing, formulation, dosing, and monitoring for outcomes that range from bacterial clearance to immune reactions. Clinical evidence includes case reports, small cohort studies, compassionate-use programs, and a limited number of randomized controlled trials, each with different methodological rigor. An information source that conflates experimental findings with established practice, or that presents anecdotal successes as generalizable results, could mislead clinicians in consequential ways.

The evaluation framework used in the study mirrors the standards applied to other emerging medical information tools. Responses generated by the models were assessed for factual accuracy against the primary literature, for completeness in covering the essential elements of a clinical question, for internal consistency, and for the presence of appropriate caveats and safety information. Questions posed to the models spanned the practical spectrum of phage therapy: indications for use, the process of matching phages to bacterial pathogens, dosing and route of administration, known adverse effects, interactions with antibiotics, regulatory status, and the strength of the clinical evidence base. This breadth matters because a model might perform well on general background questions while failing on the specific, operational details that determine whether a therapy is used correctly.

The results highlight a pattern that has emerged across evaluations of AI in medicine. Large language models generally perform well on questions with abundant, well-established answers in the training data. Basic descriptions of what bacteriophages are, how they kill bacteria, and why they are being reconsidered in the era of antimicrobial resistance tend to be accurate and clearly expressed. The models are also effective at summarizing the general rationale for phage therapy and at explaining concepts such as phage specificity and the importance of susceptibility testing. For a clinician seeking orientation in an unfamiliar field, this level of performance can be genuinely useful, providing a readable entry point that would once have required hours of literature searching.

Performance degrades, however, as questions move from general principles to specific clinical detail. The study found that models can produce answers that are partially correct but incomplete, omitting critical caveats such as the experimental status of many phage therapy protocols or the limited availability of approved phage products in most jurisdictions. Some responses blended established facts with outdated or unsupported claims, presenting them with equal confidence. In a domain where treatment decisions depend on precise, current information about phage-bacterium matching and evolving regulatory frameworks, such subtle inaccuracies are not trivial. A response that is ninety percent correct can still be clinically dangerous if the incorrect ten percent concerns dosing, safety, or the evidence supporting a therapeutic claim.

Another dimension of the evaluation concerns how the models communicate uncertainty. Trustworthy medical information sources distinguish clearly between what is proven, what is plausible, and what is speculative. The study indicates that large language models vary considerably in this respect, sometimes providing appropriate disclaimers about the experimental nature of phage therapy and sometimes presenting contested or preliminary findings as settled. This variability is itself informative, because it suggests that clinicians cannot assume a consistent standard of epistemic caution across different questions or different models. The fluency of AI-generated text can mask this inconsistency, making careful verification more important, not less.

The implications extend beyond phage therapy to the broader integration of artificial intelligence into clinical practice. The study’s authors frame their work as a caution against treating chatbots as authoritative references, particularly in specialized and rapidly evolving fields. At the same time, the findings do not support dismissing these tools outright. Used as a starting point for literature exploration, a drafting aid, or a way to formulate better questions for specialists, large language models can add real value. The critical requirement is human oversight: clinicians with domain knowledge must remain in the loop, verifying AI-generated claims against primary sources before any of that information influences patient care. This is the same standard applied to other secondary sources of medical information, and the study argues it should apply with equal force to AI.

The research also points toward what would be needed for large language models to become genuinely reliable clinical resources. Improvements are likely to come from several directions: grounding model responses in curated, up-to-date medical databases rather than relying solely on static training data; developing domain-specific evaluations that test models against expert-validated question sets; and building transparency features that allow users to trace claims back to their sources. For phage therapy specifically, a field whose evidence base is growing quickly as new trials are completed, the ability to incorporate current literature is essential. Until such systems mature, the study’s central message stands: large language models can be informative conversational partners on phage therapy, but their outputs should be regarded as provisional drafts of knowledge, subject to expert review, rather than as substitutes for the primary literature and clinical judgment on which safe patient care ultimately depends.

Subject of Research: Evaluation of large language models as sources of clinical information on bacteriophage therapy

Article Title: Performance of large language models as a source of clinical information on bacteriophage therapy

Article References: Walter, N., Amanatullah, D. F., Debarbieux, L., Doub, J. B., Ferry, T., Groß, J., Międzybrodzki, R., Mirzaei, M. K., Deng, L., Rācenis, K., Suh, G. A., Que, Y.-A., Górski, A., & Rupp, M. (2026). Performance of large language models as a source of clinical information on bacteriophage therapy. npj Viruses, 4(1), Article 41. https://doi.org/10.1038/s44298-026-00224-2

Image Credits: AI Generated

DOI: 10.1038/s44298-026-00224-2

Keywords: bacteriophage therapy, large language models, artificial intelligence, antimicrobial resistance, clinical information, medical AI, npj Viruses, hallucination, infectious diseases, phage selection, clinical decision support, evidence quality