The scale of the pressure on dermatology services provides the backdrop for the findings. Urgent suspected skin cancer referrals in England have almost tripled since 2009, yet only around 6% of those referrals result in an urgent skin cancer diagnosis. Meanwhile, approximately one in four dermatologist roles in the UK remains unfilled, creating a widening imbalance between demand and specialist capacity. Patients with benign lesions therefore often queue alongside those with aggressive malignancies, and the system strains to triage them efficiently. The new study tested whether a CE-marked Class III AI medical device could safely absorb a meaningful share of that low-risk workload without compromising patient safety.

The autonomous AI-supported pathway was introduced across two hospital sites following an initial validation period. Patients referred on urgent suspected skin cancer pathways were offered the option of AI-based assessment, and uptake was striking: 86% consented to autonomous decision-making. In total, the pathway managed 8,391 patients, representing 94% of all urgent suspected skin cancer referrals across the two hospitals. Using clinical photographs and dermoscopic images captured with smartphone-based technology, the system classified skin lesions and autonomously discharged benign cases, while routing higher-risk lesions to teledermatologists for specialist review.

The proportions of patients managed without any clinician involvement were substantial. After exclusions, the AI autonomously discharged 31% of patients at one hospital and 25% at the other without a dermatologist ever reviewing their case. Teledermatologists subsequently discharged a further 24% and 25% of patients respectively, meaning that in both settings roughly half of all referred patients left the pathway without a face-to-face specialist consultation. For a service in which one in four consultant posts is vacant, that shift in workload distribution carries obvious operational significance.

The efficiency gains extended beyond simple discharge rates. Compared with standard teledermatology, the autonomous pathway reduced the proportion of patients requiring routine follow-up from 27% to 12%. Biopsy rates also fell markedly, to 27% under the autonomous pathway compared with 43% for conventional face-to-face care, suggesting the system was not simply deflecting work but sharpening the selection of lesions that genuinely warranted invasive investigation. Fewer unnecessary biopsies mean fewer procedures, less patient anxiety and lower costs across the pathway.

Aggregated across the 16-month study period, the researchers estimated that the autonomous pathway saved 2,851 hours of clinician time relative to a traditional face-to-face model, an approximate 62% gain in clinical capacity. Based on standard 20-minute consultations, that equates to more than 8,500 additional face-to-face appointments across the two hospitals. Dr Lucy Thomas, the lead author and a consultant dermatologist at Chelsea & Westminster Hospital NHS Foundation Trust, argued that the true value of the technology lies in what it frees specialists to do. “We believe the greatest value of autonomous AI lies not in the technology itself, but in the specialist capacity it unlocks,” she said. “Every hour saved reviewing low-risk lesions can be reinvested in patients with skin cancer, helping them access timely treatment to improve prognosis, and in patients with severe inflammatory skin disease, where earlier access to specialist care and effective treatments can transform quality of life.”

Safety monitoring was a central component of the deployment, and the results offer some reassurance alongside important caveats. In a national dataset incorporating both study sites, the system’s sensitivity exceeded 98% for invasive melanoma, squamous cell carcinoma and basal cell carcinoma, with a specificity of 72.1%. High sensitivity is the critical metric in a cancer pathway, since missed malignancies carry the gravest consequences. Nevertheless, surveillance identified six false-negative cases that had been discharged by the pathway: five basal cell carcinomas and one melanoma in situ. These were detected through post-market surveillance, and no adverse outcomes were identified within the available follow-up period, but the figures underline that autonomous discharge is not error-free.

That experience shaped the study team’s central lesson about deploying medical AI responsibly. “One of the key lessons for us is that deploying an AI system safely isn’t a one-off exercise,” Dr Thomas said. “You need to keep monitoring it, understand when things go wrong, learn from those cases and make sure patients themselves know what to look out for.” The emphasis on continuous post-market surveillance reflects a broader shift in thinking about AI as a medical device: unlike a static drug with a fixed safety profile, an algorithm’s performance can drift with changing populations, imaging practices and referral patterns, demanding ongoing audit rather than a single approval milestone.

The study also highlights the practical mechanics of making such a pathway work at scale. Consent was sought from every patient offered autonomous assessment, and the high acceptance rate of 86% suggests that, when the process is explained clearly, patients are willing to trust algorithmic discharge decisions for low-risk lesions. The device itself, funded by Chelsea & Westminster Hospital NHS Foundation Trust and partly supported by a grant from La Roche-Posay, operates on images that can be captured with smartphone-based dermoscopy, lowering the infrastructure barrier for adoption in community settings and smaller clinics where dedicated hospital imaging equipment is unavailable.

The researchers are careful to frame the findings as a proof of concept rather than a universal prescription. “If these findings are replicated across larger populations and different healthcare settings, autonomous AI could become an important part of creating a more sustainable dermatology service – not by replacing dermatologists, but by allowing scarce specialist expertise to be focused where it can make the greatest difference to patients’ lives,” Dr Thomas concluded. With referral volumes still climbing and workforce shortages showing no sign of easing, the study offers one of the most concrete demonstrations to date that autonomous AI can operate safely inside a cancer pathway while returning thousands of hours of specialist time to the patients who need it most.

Subject of Research: Autonomous AI triage of urgent suspected skin cancer referrals in dermatology

Article Title: Autonomous AI could create capacity for thousands more dermatology appointments, real-world study finds

Article References: Autonomous AI could create capacity for thousands more dermatology appointments, real-world study finds. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: autonomous AI, dermatology, skin cancer, teledermatology, AI medical device, melanoma, basal cell carcinoma, NHS, clinical capacity, EADV Congress 2026, post-market surveillance, dermoscopy

The study, led by researchers at University Hospital Würzburg together with the system’s developer and the Parkinson Stiftung, the non-profit German foundation that commissioned and operates the service, is being hailed as a template for what regulators have long demanded but rarely seen: structured, post-deployment monitoring of an autonomous, unsupervised, publicly accessible medical AI. Unlike earlier evaluations that relied on physician supervision of every exchange or on user surveys, this analysis covered every conversation from the first day of routine operation, with no recruitment, incentives, or modification of user behaviour. Users asked genuinely unprompted questions about symptoms, medications, daily coping, and therapies—precisely the conditions under which the failure modes that matter most become visible.

jAImes was deliberately engineered as a counter-design to the general-purpose chatbots that have fared poorly in recent audits. Rather than allowing a large language model to answer freely, the system separates a user-facing advisory agent, built on Claude Sonnet 4.5, from a database agent running Mistral Small 3.2, which synthesises answers strictly from a curated knowledge base of 221 documents comprising peer-reviewed literature, expert lectures, and validated patient-education materials. A retrieval-augmented pipeline with hybrid search, fusion, and reranking grounds the answers in these sources, while system-level prompts explicitly prohibit diagnostic interpretation, medication dosing, and therapy modification, and define emergency triggers that are supposed to activate a crisis protocol focused on supportive language and signposting to emergency services. The tool is explicitly scoped as a non-diagnostic, non-therapeutic information system, free to use in Germany without registration or advertising.

To evaluate the deployment, the team introduced a new quality-assurance framework called CARE-LLM, short for Conversation-level AI Real-world Evaluation, described for the first time in this study. It combines four components: comprehensive automated triage of every conversation, structured human expert review of flagged cases, sampling-based validation of conversations the triage rated as adequate, and a feedback loop that routes confirmed failures to class-specific remediation. Automated screening classified 88.6 percent of conversations as good, 11.0 percent as partially adequate, and just 0.4 percent as inadequate, with knowledge-base gaps rather than unsafe answers accounting for most partial ratings. Triage flagged 212 conversations, and together with 28 negative-feedback exchanges, 224 unique conversations entered structured review; 45 warranted detailed specialist assessment, and five were confirmed critical by board-certified neurologists with more than a decade of movement-disorder experience, all of whom were structurally independent of the developer and the foundation.

Those confirmed events fell into three distinct failure classes that the authors propose as an empirically grounded taxonomy. Knowledge boundary failures occur when user queries exceed the system’s retrieval coverage and the generative component fills the gap with confident output instead of signalling uncertainty—one such case involved factually incorrect information about an ongoing clinical trial and a misstated investigator affiliation. Robustness failures reflect susceptibility to manipulative or adversarial prompting, illustrated by a conversation interrupted by the underlying content-safety filter in a pattern consistent with a jailbreak-style attack. Escalation failures are breakdowns in predefined emergency responses: of three conversations containing explicit suicidal ideation, two surfaced emergency contacts but were judged insufficiently aligned with the intended protocol, and one—the exchange following a fresh Parkinson’s diagnosis—triggered no crisis response at all.

Perhaps the most consequential finding came from the framework’s third component, which exists precisely to expose the blind spots of automated triage. When the researchers randomly sampled 100 of the 1,803 conversations classified as adequate and subjected them to independent clinical review, four contained clinically critical errors—a conditional false-negative rate of 4 percent within the good stratum. These were not exotic failures but classic knowledge boundary errors: an inappropriate recommendation of memantine for Parkinson’s disease dementia, a pharmacologically inaccurate statement about the duration of prolonged-release levodopa, a non-first-line suggestion of amantadine for tremor, and a medication misidentification in which a levodopa/carbidopa preparation was labelled as selegiline. The authors stress that this figure is a single-window estimate with a wide confidence interval, spanning roughly one missed critical event per 60 to one per 10 good-rated conversations, and that it cannot be extrapolated to the full corpus. But its message is unambiguous: favourable automated quality metrics can coexist with a clinically meaningful residual rate of critical failures invisible to those metrics.

The usage data themselves offer a portrait of who turns to such systems and what they want to know. Usage was continuous, averaging 47.3 messages per day, with a median response latency of 33 seconds and knowledge-base retrieval triggered in 91.9 percent of conversations. Where users disclosed their role, 75 percent were patients, 16 percent relatives or caregivers, and the remainder physicians and nursing staff; the most common age band was 60 to 69 years, matching the intended population. The dominant themes were symptoms, daily life and coping, medications, and therapies. Explicit knowledge gaps appeared in 12.3 percent of conversations, most often concerning region-specific contacts such as local self-help groups and specialist clinics, newer medications, current studies, and procedures like high-intensity focused ultrasound. These gaps rarely produced unsafe answers but limited completeness and local usefulness, underscoring that content maintenance is itself a safety function.

The authors are candid that the evaluation is an operator-led post-market assessment embedded in an imperfect deployment. jAImes entered public use without a formal pre-deployment validation study; the informal six-month expert-testing phase, which included adversarial jailbreak-style probing, was formative rather than evaluative, and none of the critical failures reported here was caught before launch. The team explicitly rejects the notion that deployment-with-surveillance can substitute for pre-deployment validation—the missed suicidality escalation occurred in a live system, and no retrospective monitoring result can remove that exposure. They also acknowledge that anonymity, while lowering the threshold for disclosing stigmatised concerns and enforcing GDPR data minimisation, made individual follow-up impossible and ruled out any real-time clinical monitoring or emergency intervention during deployment. The crisis-response protocol was revised at the first scheduled maintenance after the analysis identified the missed escalation, and re-testing confirmed the specific failure was remedied, though the system configuration has since changed in other respects as well.

Notably, an external legal assessment concluded that jAImes does not qualify as a medical device under the EU Medical Device Regulation, because its purpose is confined to general disease information, nor as a high-risk AI system under the EU AI Act—yet the surveillance reported here was adopted voluntarily, echoing the life-cycle monitoring principles of the AI Act’s Article 72 and FDA postmarket guidance. The study’s regulatory argument is pointed: clinically consequential failures arise in patient-facing information tools regardless of device status, and a conventional usability or satisfaction study would have detected none of the critical events documented. The authors argue that prospective, conversation-level evaluation with independent clinical adjudication should be treated as a design requirement for patient-facing medical large language models, and that periodic sampling-based validation should complement flag-driven review as routine deployment-level quality assurance.

The broader significance extends well beyond Parkinson’s disease. Recent randomised trials of patient-facing large language models in digital psychotherapy and primary-to-specialist care transitions measured clinical efficacy, not post-deployment safety, and audits of unscoped consumer chatbots have rated roughly half of health-related responses as problematic, with models hallucinating citations. jAImes shows that tightly scoped, retrieval-augmented design can dramatically improve on that baseline while still harbouring residual failure modes that only real-world, clinician-adjudicated surveillance can reveal. The team cautions that the framework’s transferability to other domains and systems requires independent replication, and that fairness auditing—particularly for older, non-German-speaking, or digitally excluded populations—remains a priority. But the central lesson stands: safety properties of medical AI must be specified, monitored, and governed across the entire operational life cycle, because even the most carefully designed safeguards are themselves objects that demand ongoing evaluation.

Subject of Research: Post-deployment safety surveillance of a publicly deployed generative AI chatbot providing Parkinson's disease information to patients and caregivers

Article Title: Real-world use and evaluation of a generative AI chatbot for Parkinson's disease information: a prospective observational study

Article References: Lange, F., Mardi, S., Binder, T., Reich, M. M., Odorfer, T., & Volkmann, J. (2026). Real-world use and evaluation of a generative AI chatbot for Parkinson's disease information: a prospective observational study. The Lancet Regional Health – Europe, 70, Article 101866. https://doi.org/10.1016/j.lanepe.2026.101866

Image Credits: AI Generated

DOI: 10.1016/j.lanepe.2026.101866

Keywords: Parkinson's disease, generative AI chatbot, large language models, post-market surveillance, patient safety, retrieval-augmented generation, clinical adjudication, CARE-LLM framework, digital health, medical misinformation, suicidality escalation, EU AI Act