Doctors Trust AI Consultation Advice as Much as Human Advice, Study Finds

When a physician faces a complicated clinical question, the traditional route has been to seek a specialist’s opinion. Increasingly, however, clinicians are turning to generative artificial intelligence tools for that same kind of consultation, and a new randomized study suggests they rate the machine’s advice just as highly as a human expert’s, and follow it just as often. The finding, published in the Journal of General Internal Medicine, raises provocative questions about how AI is quietly reshaping medical decision-making.

Researchers at the University of Pittsburgh School of Medicine recruited 44 internal medicine teaching faculty from a single large academic medical center and asked them to review four clinical vignettes, each describing a patient scenario with a management decision to be made. For every case, the participants read a piece of electronic consultation advice, or e-consultation, generated by OpenEvidence, a medically specialized generative AI system that has rapidly become one of the most widely used clinical AI platforms among physicians. Before and after reading the advice, the physicians answered multiple-choice questions about how they would manage the patient, allowing the investigators to measure whether the AI’s recommendations actually changed their decisions.

The study’s central twist was a randomization scheme. Half of the participants were told the advice had been written by a specialist attending physician, while the other half were told it came from a medically specialized artificial intelligence system. In reality, all of the advice was AI-generated. The design allowed the researchers to isolate whether the label alone, the perceived source of the advice, influenced how physicians judged its quality or whether they adopted its recommendations.

The results were striking in their uniformity. Across six domains of perceived quality, completeness, actionability, clarity, appropriateness, trustworthiness, and use of evidence-based practice, the physicians gave the advice high marks, with overall mean ratings ranging from 4.2 to 4.7 on a 5-point scale. Critically, there were no statistically significant differences between the group that believed the advice came from a human specialist and the group that knew it came from an AI system. The label, in other words, did not matter.

The adoption data were even more dramatic. Before reading the e-consultation advice, only 37.5 percent of participants chose the management option that the AI would later recommend. After reading it, that figure jumped to 81.8 percent, an odds ratio of 8.43 with a 95 percent confidence interval of 5.00 to 14.28 and a p-value below 0.001. Whether the physicians thought they were reading the words of a human colleague or a machine made no measurable difference in this shift, with an odds ratio of 1.18 for the labeling effect and a confidence interval spanning 0.51 to 2.75.

These findings arrive at a moment when generative AI tools are being integrated into clinical workflows at unprecedented speed. OpenEvidence, the platform used in the study, has been described by its own developers as the fastest-growing application for physicians in history, following a funding round that valued the company at 3.5 billion dollars. Prior research has shown that large language models can perform at expert level on complex diagnostic challenges, and earlier studies have documented that clinicians’ diagnostic reasoning can be swayed by AI input, sometimes for better and sometimes for worse. What this new study adds is direct evidence that experienced teaching faculty, the physicians responsible for training the next generation, rate AI-generated consultation advice as highly as they would rate human specialist advice, and change their management decisions accordingly.

The context of e-consultation matters here. Electronic consultations, in which a primary care clinician submits a question to a specialist asynchronously rather than referring the patient for an in-person visit, have expanded rapidly as a means of improving access to specialty expertise. Systematic reviews have documented their benefits for connecting primary care to hospital-level expertise, but studies have also identified suboptimal e-consult requests and variable adherence to specialist recommendations. The Pittsburgh study suggests that when the consultant is an algorithm rather than a human specialist, physicians may be just as receptive, and just as likely to act on what they read.

That receptiveness cuts both ways. On one hand, if AI-generated advice is accurate, well-referenced, and genuinely useful, high adoption rates could accelerate the dissemination of best practices and reduce variation in care. Prior work has shown that management changes following multidisciplinary team discussions are associated with improved survival outcomes for cancer patients, and that second opinions in general internal medicine frequently alter diagnoses and management plans. If an AI system can approximate the value of a specialist consultation, the implications for access and efficiency are enormous, particularly in underserved areas where specialist availability is limited.

On the other hand, the study’s authors and the broader literature caution that susceptibility to AI recommendations is not inherently benign. Research on clinical decision support has shown that clinicians can be overly compliant with algorithmic advice, a phenomenon sometimes called automation bias, and recent work has demonstrated that the correctness of AI recommendations directly affects diagnostic accuracy in clinical decision-making. If physicians adopt AI-generated advice at rates approaching 82 percent regardless of its source, the quality assurance burden shifts from the individual clinician to the system itself. The study did not evaluate whether the AI’s recommendations were objectively correct for each vignette, only whether physicians found them persuasive and acted on them.

The researchers also note that their participants were teaching faculty at a single academic medical center, a group that may be more familiar with AI tools than the average clinician, and that the vignettes were hypothetical rather than live patient cases. Whether the same patterns hold in real clinical settings, with real patients and real time pressures, remains an open question. The authors call for future studies to explore what factors influence the adoption of AI-generated recommendations and how curricula for faculty and trainees can shape consultation utilization. As generative AI becomes embedded in the daily workflow of medicine, understanding not just what these systems say but how physicians respond to them may prove as important as the underlying technology itself.

Subject of Research: Physician perceptions and adoption of AI-generated e-consultation advice in clinical decision-making

Article Title: Physician Ratings and Adoption of AI-Generated e-Consultation Advice: A Randomized Clinical Vignette Study

Article References: Desjardins, G., Jain, V., Simonson, M. G., & McQuade, C. N. (2026). Physician Ratings and Adoption of AI-Generated e-Consultation Advice: A Randomized Clinical Vignette Study. Journal of General Internal Medicine. https://doi.org/10.1007/s11606-026-10777-w

Image Credits: AI Generated

DOI: 10.1007/s11606-026-10777-w

Keywords: generative artificial intelligence, e-consultation, clinical decision-making, internal medicine, OpenEvidence, randomized vignette study, physician perceptions, medical education, automation bias, clinical reasoning, Journal of General Internal Medicine, healthcare AI