The research, led by Vinícius Vicente Soares and colleagues at the Federal University of Health Sciences of Porto Alegre in Brazil, describes the technical development and a pilot expert-based content validation of an interactive voice prototype built on GPT-4o, OpenAI’s multimodal model. Rather than testing educational outcomes in a cohort of students—a step the authors explicitly defer to future work—the study documents how such a system is actually engineered, and what a seasoned psychiatrist found when she sat down to interview its virtual patients. The result is both a proof of concept and an unusually candid inventory of the technology’s shortcomings.

The team built four psychiatric personas covering conditions chosen for their epidemiological weight and their semiological richness: major depressive disorder, bipolar disorder in a manic episode, schizophrenia, and attention-deficit/hyperactivity disorder. Together these span the core domains a trainee must probe—mood, thought, and attention—and demand very different interviewing styles, from drawing out a psychomotorically slowed, withdrawn depressed patient to keeping pace with the flight of ideas of mania. Initial profiles were drafted by the researchers from DSM-5-TR and ICD-11 criteria and established psychopathology literature, with the GPT-o3 model used strictly as a writing assistant to polish cohesion, a deliberate design choice meant to keep the clinical content under human control.

Each persona then went through a structured human-in-the-loop refinement process. A board-certified psychiatrist with more than fifteen years of experience in medical education—the study’s subject matter expert—conducted test interviews with each virtual patient, and her qualitative feedback drove successive revisions of the system prompts. Three refinement cycles per persona were completed, adjusting symptomatological nuances such as increasing response latency for the depressed patient and toning down caricature-like behaviors, until the expert judged each profile clinically plausible for a training environment.

The prompt engineering, described in unusual technical detail, is arguably the study’s most useful contribution. Each persona was operationalized through a master instruction with standardized components: a contextualization block defining the patient’s identity and framing the encounter as a first psychiatric assessment; a core identity section containing a biographical summary, central symptomatology with concrete verbal and behavioral examples, the predominant affective state that directly informs the text-to-speech prosody, and the patient’s level of insight into their condition; and dynamic interaction guidelines specifying vocal style, speech rhythm, and reactivity—so that the virtual patient might open up to an empathic interviewer but grow defensive when delusional beliefs were directly challenged.

Equally important were the constraints. So-called negative prompts were written to suppress known failure modes of large language models: hallucination of facts, the reflex to be overly helpful, drift into generic AI-assistant behavior, and the temptation to use technical psychiatric jargon—which would let a trainee off the hook of translating a patient’s own words into semiological terms. The prompt also embedded a formative feedback mechanism: on the command “END,” the model generates a structured critique of the interviewer’s performance, identifying the semiological phenomena displayed by the persona, evaluating the interviewer’s approach, and suggesting improvements, such as exploring suicide risk more thoroughly.

Evaluation followed a heuristic evaluation protocol adapted from usability engineering. For each persona, the expert received only a brief case summary—a fictitious name, age, and chief complaint—without access to the underlying prompt, then conducted a free, unstructured thirty-minute interview that was fully recorded. She rated each simulation on Likert scales covering clinical fidelity, persona consistency, affective expression, interaction quality, and pedagogical value, and her open-ended comments were subjected to thematic content analysis. The authors acknowledge a key limitation here: the evaluator was involved in the iterative development of the personas, so the assessment was not fully independent, and the study rests on a single expert rather than a panel.

The quantitative picture was strikingly consistent. The pedagogical value dimension received the maximum score of five across all four simulations, suggesting the expert saw genuine educational potential regardless of clinical imperfections. The ADHD persona earned the best overall rating at 4.4, while the schizophrenia persona scored lowest at 3.8. Fidelity-related dimensions, particularly interaction with the interviewer, lagged behind, and the item assessing whether the simulated patient avoided stereotyping received the minimum score for three of the four personas—a red flag that shaped much of the qualitative analysis.

Three themes dominated the expert’s critique. First, clinical stereotyping: the virtual patients tended to deliver textbook presentations, announcing their symptoms with a clarity real patients rarely muster. Of the schizophrenia persona, the evaluator observed that although the patient described typical symptoms in a detailed and coherent manner, she was very stereotyped, since patients with schizophrenia generally do not reveal their symptoms so readily. Second, vocal naturalness fell short: the manic patient talked a great deal but lacked the characteristic pressure of speech of true mania, stopping in ways the evaluator found unnatural, and the artificial prosody undermined immersion. Third, platform barriers intruded: content moderation on the commercial ChatGPT platform censored discussion of hypersexuality in the mania persona—a core symptom—producing an artificially aseptic scenario, and the model could not reproduce regional linguistic traits, a limitation scored at the minimum for every persona.

The feedback mechanism, by contrast, performed well, generating structured and pertinent analyses that correctly identified phenomena such as flight of ideas and expansive mood. The authors’ conclusion is measured: voice-based LLM simulations are technically feasible and pedagogically promising, but current limits in multimodal realism, the risk of teaching students to recognize caricatures rather than the heterogeneous face of mental illness, and the opacity of these models mean the tool is best suited for supervised formative practice, not high-stakes assessment. As a proof of concept, the study maps the terrain ahead—better prompt engineering, dedicated platforms, multi-center trials with real trainees, and rigorous faculty oversight—before AI patients can take a lasting seat in the psychiatry classroom.

Subject of Research: Development and expert-based content validation of a GPT-4o voice prototype for simulating psychiatric patient interviews in medical education

Article Title: LLM-Based Psychiatric Interview Simulation: Technical Development and Pilot Expert-Based Content Validation of a Voice Prototype

Article References: Soares, V. V., Passos, F. F. D. C., Shansis, F. M., & Herbert, J. S. (2026). LLM-Based Psychiatric Interview Simulation: Technical Development and Pilot Expert-Based Content Validation of a Voice Prototype. Academic Psychiatry. https://doi.org/10.1007/s40596-026-02422-9

Image Credits: AI Generated

DOI: 10.1007/s40596-026-02422-9

Keywords: large language models, GPT-4o, psychiatric education, virtual patients, clinical simulation, psychiatric semiology, prompt engineering, medical education, voice interface, mental status examination, content validation, artificial intelligence