The researchers set out to map where, in the machinery of modern healthcare delivery, AI-related harms actually occur. Rather than treating AI as a single black box that a doctor either accepts or rejects, they categorized the cases according to the specific clinical tasks physicians perform: diagnosing conditions, ordering and interpreting tests, planning treatment, triaging patients, documenting and communicating information, and coordinating care across institutions. The distribution was revealing. Harms clustered not only around diagnostic tools — the category that dominates public imagination and much of the regulatory conversation — but also around administrative and coverage systems, including AI used by insurers to deny or curate care, and screening tools deployed by state agencies to flag families for investigation.

That breadth matters because it exposes a structural blind spot in both technology design and legal doctrine. Much of the existing scholarship on AI liability in medicine has centered on a deceptively simple question: if an AI tool outperforms a doctor, does reliance on it constitute negligence, or does deviation from it? Prior work, including widely cited analyses in JAMA and law reviews, has debated whether physicians could face liability for ignoring a competent algorithm or for deferring to a flawed one. Mansi and Riedl argue that this physician-centered frame is too narrow to describe what actually happens when patients are harmed. An insurance company’s algorithm may terminate home-care coverage; a state screening tool may incorrectly flag a parent for child-welfare investigation; a triage system may misdirect emergency care. In each scenario, the patient’s injury emerges from decisions distributed across an organization, with the physician often having little visibility into — or control over — how the AI was configured, trained, deployed or applied.

The legal cases the researchers analyzed come from a meticulous triangulation of three public databases: the Database of AI Litigation maintained at George Washington University Law School, the Health Litigation Tracker, and the AI, Algorithmic and Automation Incidents and Controversies repository, a community-maintained archive of documented harms. As of February 2025, all three resources are freely accessible online, and the researchers referenced individual cases through unique identifiers, centering documented harms rather than the specific systems involved. The case-study approach allowed the authors to reconstruct, case by case, how harm materialized, which stakeholders were positioned between the patient and the algorithm, and what barriers the patient encountered in seeking recourse.

Those barriers, the study finds, are both legal and technological, and they reinforce one another. On the legal side, liability structures remain tethered to a model of medical malpractice that presumes a human decision-maker with enough information and discretion to have acted differently. When an algorithm’s output is embedded in a payer’s coverage workflow or a state agency’s screening pipeline, a patient’s lawyer must identify which entity’s conduct was negligent, prove causation, and often overcome trade-secret protections that keep the model’s logic opaque. On the technological side, the tools themselves are rarely designed for contestability. Explanations, where they exist, are built for clinicians — and even for clinicians, a substantial body of human-computer interaction research shows that explainability features are frequently underused, misunderstood or shaped more by regulatory checkbox than by genuine decision support. They are almost never built for the people who most need them when things go wrong: patients and their advocates.

Here the study makes its most provocative move. Mansi and Riedl argue that lawyers working for patients should be recognized as legitimate users of AI systems — and, in a sense, as intermediaries whose needs deserve to shape the design of medical AI from the ground up. If an explainability interface cannot help a patient’s legal team understand which stakeholder made which consequential decision, what data the model relied on, and where in the pipeline the error propagated, then the tool has effectively insulated everyone but the patient from scrutiny. The authors connect this to a growing research program on legally-informed explainable AI, which aims to produce explanations aligned with the evidentiary demands of legal proceedings rather than with the aesthetic of transparency that currently dominates industry practice.

The researchers also draw a pointed lesson from documented incidents involving coverage algorithms. Reporting on how Medicare Advantage plans have used AI tools to cut off care for seniors, alongside legal scholarship on regulating healthcare coverage algorithms, illustrates the pattern the study identifies: harms arise at the intersection of commercial incentives, algorithmic automation and fragmented oversight. A patient denied post-acute care by an algorithmic coverage tool may appeal through administrative channels, sue, or simply suffer silently. Each path demands different evidence, different expertise and different institutional access — resources that are unevenly distributed among the very populations most likely to be harmed.

What can be done? The authors propose reform on two fronts. First, liability structures should evolve to reflect the distributed reality of AI-mediated care. They point to legal scholarship exploring theories such as the common enterprise theory of liability, which could allow responsibility to be shared across the ecosystem of actors — developers, payers, facilities, regulators — that jointly shape how a tool affects patients. Distributed governance frameworks for medical AI, and regulatory thinking that treats software as a medical device within a whole system rather than as a standalone artifact, offer templates for how accountability could be allocated more realistically. Second, AI tools for healthcare should be designed for contestability by default: systems should log which stakeholders shaped their outputs, generate explanations usable in legal contexts, and support advocates — including patients’ lawyers — in reconstructing what happened when harm occurs. Contestable AI, an emerging design philosophy in human-computer interaction, offers concrete principles: make decisions reversible, make the grounds of decisions accessible, and make the pathway to human review visible to affected people.

The stakes of getting this wrong extend beyond individual lawsuits. If patients cannot seek recourse, there is no corrective feedback signal pushing developers and deployers to fix flawed systems; courts become the only place where the accumulated evidence of algorithmic failure can surface, and only for those few patients wealthy or persistent enough to get there. Conversely, if accountability is designed into the system — technologically through auditable, contestable tools, and legally through liability frameworks that track the actual distribution of decision-making power — then recourse becomes possible without requiring every patient to become a litigant. The study’s authors, whose work was supported in part by the National Science Foundation, frame this as a shift from physician-centered to patient-centered accountability, a reorientation that demands new collaboration between lawyers, technologists, clinicians and regulators.

For a healthcare system rapidly absorbing AI into triage, diagnosis, documentation and coverage decisions, the message of the litigation record is uncomfortable but clear: the technology is arriving faster than the accountability structures needed to govern it, and the people bearing the consequences are the ones the current framework serves least. Mansi and Riedl’s analysis of 31 cases turns courtroom silence into a design specification — a map of where AI harms happen in healthcare delivery, and a blueprint for tools and laws that could finally let patients fight back.

Subject of Research: Analysis of US litigation involving medical AI tools and patient-centered accountability for AI-related harms in healthcare delivery.

Article Title: Implications of current litigation on the design of AI tools for healthcare delivery and related legal frameworks

Article References: Mansi, G., & Riedl, M. (2026). Implications of current litigation on the design of AI tools for healthcare delivery and related legal frameworks. Nature Health. https://doi.org/10.1038/s44360-026-00186-y

Image Credits: AI Generated

DOI: 10.1038/s44360-026-00186-y

Keywords: medical AI, healthcare litigation, AI liability, patient recourse, explainable AI, contestable AI, healthcare delivery, algorithmic harms, coverage algorithms, legal frameworks, patient-centered accountability, Nature Health