When the Model Is Not the Product: Teaching Clinicians to Read AI Evidence

A large language model that scores impressively in a research benchmark is not the same thing as the chatbot a patient opens on their phone at two in the morning. That gap between model and product sits at the heart of a new commentary published in Academic Psychiatry by Matthew Flathers, Phuong Anh Nguyen, Haruka Notsu, Julian Herpertz, Sean J. Ryan and John Torous of Beth Israel Deaconess Medical Center, with Herpertz additionally affiliated with Charité Universitätsmedizin Berlin. The paper, titled From Model to Product: Understanding and Teaching AI Evidence, argues that clinicians, educators and trainees routinely conflate two distinct layers of artificial intelligence evidence, and that this confusion has real consequences for how psychiatry evaluates, adopts and teaches these tools.

The distinction the authors draw is technical but consequential. A model is the underlying neural network, a set of weights trained on data, whose behavior can be probed under controlled conditions. A product is what a user actually encounters: the model wrapped in a system prompt, safety filters, retrieval tools, guardrails, versioning policies and a user interface, all of which can change its outputs dramatically. Evidence generated on one layer does not automatically transfer to the other. A randomized trial of GPT-4 assistance improving physician performance on patient care tasks, such as the one published in Nature Medicine by Goh and colleagues in 2025, tells clinicians something about model capability under study conditions, but it does not tell them what a commercial chatbot product will say to an individual user tomorrow, because the product layer can be updated, constrained or reconfigured at any time without notice.

To make this abstract point concrete, the team conducted an illustrative comparison, reproduced in the paper’s figures, in which the same message was sent to a model through different access routes. The comparison used Claude Opus 4.6 as its object of study, with the outputs reproduced as study material rather than as authored text. The exercise demonstrated that identical prompts routed through different interfaces of the same underlying system can yield meaningfully different responses, a finding that undermines the common practice of citing a model name as if it fully specifies the intervention being evaluated. The authors are explicit that the illustrative messages were authored by the study team and contained no patient information, and that the work did not involve human subjects and required no ethics committee approval.

The timing of this argument matters. Large language models have moved rapidly from laboratory curiosities into real clinical workflows. A 2025 systematic review by Artsi and colleagues in Frontiers in Digital Health catalogued applications and implementation challenges of these models in real-world care, while ambient AI scribes have been deployed at scale to alleviate the burden of clinical documentation, as reported by Tierney and colleagues in NEJM Catalyst. Meanwhile, meta-analytic work such as that of Wei and colleagues in the Journal of Biomedical Informatics has attempted to synthesize the quality of ChatGPT-generated medical responses, and studies like Rutledge’s in Learning Health Systems have probed the diagnostic accuracy of GPT-4 on both common clinical scenarios and challenging cases. Each of these evidence streams answers a slightly different question, and the commentary contends that medical education has not yet equipped trainees to tell those questions apart.

Compounding the problem is the fact that the systems under study are moving targets. Research by Chen, Zaharia and Zou in the Harvard Data Science Review documented how ChatGPT’s behavior changed over time, and work presented at EMNLP 2025 by Wu and colleagues examined how medical knowledge drift and internal conflicts emerge in large language models. A model evaluated in January may behave differently by June, not because anyone retrained it in public view but because providers continuously update their systems. The commentary points to the public documentation that major AI companies now release, including OpenAI’s safety best practices and model specification and Anthropic’s system cards and published system prompts, as underused sources of evidence about the product layer. These documents reveal, for instance, how a deployed chatbot is instructed to behave, what topics it is steered away from, and what wellbeing-oriented interventions it may apply, none of which is visible in a bare model evaluation.

Nowhere is the model-product gap more charged than in mental health. A systematic review by Hua and colleagues in World Psychiatry traced the evolution of AI mental health chatbots from rule-based systems to large language models, and a 2025 paper in the Journal of Participatory Medicine by Parks and colleagues issued a direct call for critical evaluation of whether generative AI mental health chatbots are safe and evidence-based. Members of the same Beth Israel team have previously published a functional typology of large language model-associated psychotic phenomena in The Lancet Digital Health, underscoring that the clinical stakes of poorly characterized AI products are not hypothetical. When a patient interacts with a consumer chatbot about suicidal thoughts or psychotic symptoms, the relevant evidence is about the product, with all its guardrails and system prompts, yet most published evaluations concern the model.

The commentary does not leave clinicians without tools. It situates the model-product distinction within the growing ecosystem of AI-specific reporting guidelines, including the CONSORT-AI extension for clinical trials of AI interventions published in Nature Medicine by Liu and colleagues, the DECIDE-AI guideline for early-stage clinical evaluation of AI-driven decision support systems published in The BMJ by Vasey and colleagues, and the Chatbot Assessment Reporting Tool, or CHART, with its explanation and elaboration paper in The BMJ by the CHART Collaborative. These frameworks push researchers to specify exactly what was evaluated, under what configuration, at what time, which is precisely the information needed to distinguish model evidence from product evidence. Work on establishing best practices in large language model research, such as the repeat-prompting methodology paper by Gallo and colleagues in the Journal of the American Medical Informatics Association, reflects the same impulse toward rigor in a field where evaluation methods are still being invented.

For educators, the paper’s implications are practical. Teaching AI evidence in psychiatry, the authors suggest, should mean teaching trainees to ask which layer a study describes, whether the reported configuration still exists, what the vendor’s documentation says about the deployed system, and whether the outcome measured, be it benchmark accuracy, physician performance or patient-facing advice quality, corresponds to the use case at hand. It should also mean acknowledging the field’s own entanglement with these tools. In a notable act of transparency, the authors disclose that they used Anthropic’s Claude to assist with copyediting and proofreading of author-written text and to generate the HTML and CSS code used to lay out the paper’s figures, while stating that no scientific content, argument or reference was generated by AI and that all AI-assisted output was reviewed and edited by the authors, who take full responsibility for the manuscript.

The deeper message is epistemological rather than technological. Evidence in medicine derives its authority from stability: a drug’s molecular structure does not change between the trial and the prescription. Large language model products break that assumption, because the intervention itself is mutable, opaque at the product layer and governed by corporate update cycles that operate on timescales incompatible with peer review. The commentary, received in June 2026, accepted in September and published on 15 October 2026 in Academic Psychiatry, arrives as health systems worldwide are signing procurement contracts for AI tools whose evidence base may describe a different version of the product than the one that ships. Teaching clinicians to read AI evidence, the authors argue, therefore requires more than statistical literacy; it requires a new habit of mind that treats every claim about an AI system as a claim about a specific model, in a specific configuration, at a specific moment, and asks, before anything else, whether that is the same thing patients and clinicians will actually use.

Subject of Research: The distinction between AI model evidence and AI product evidence in psychiatric education and clinical evaluation

Article Title: From Model to Product: Understanding and Teaching AI Evidence

Article References: Flathers, M., Nguyen, P. A., Notsu, H., Herpertz, J., Ryan, S. J., & Torous, J. (2026). From Model to Product: Understanding and Teaching AI Evidence. Academic Psychiatry. https://doi.org/10.1007/s40596-026-02450-5

Image Credits: AI Generated

DOI: 10.1007/s40596-026-02450-5

Keywords: artificial intelligence, large language models, psychiatry, medical education, AI evidence, mental health chatbots, clinical decision support, reporting guidelines, model drift, digital health, AI safety, Claude Opus 4.6