Medical chatbots resist unnecessary tests, but waver where doctors disagree

Large language models generally identified unnecessary medical tests and treatments in a peer-reviewed study of clinical scenarios drawn from the Choosing Wisely campaign. Performance was less consistent in two areas where the clinical boundary is disputed: routine electrocardiograms in low-risk patients and ultrasound of small, incidentally discovered thyroid nodules. Researchers from Clalit Health Services and Ben-Gurion University of the Negev, with a collaborator at Rabin Medical Center–Beilinson Hospital, published the study in Annals of Internal Medicine.

The experiment examined a central difficulty in clinical practice: patients may request additional imaging, laboratory tests or antibiotics “just to be safe,” but more testing is not always better medicine. The research release notes that unnecessary tests can uncover incidental, harmless findings that lead to anxiety, specialist referrals, invasive biopsies and surgery, while an overly conservative system risks failing to act when an intervention is genuinely needed. The study therefore assessed two kinds of error. “Overuse” meant recommending a test or treatment when it was unnecessary; “false restraint” meant recommending that care be withheld when it was medically indicated.

This paired approach allowed the researchers to test whether the systems could distinguish between similar cases with opposite clinical requirements. To build the evaluation, the team used recommendations from Choosing Wisely, an international initiative founded in 2012 by the American Board of Internal Medicine Foundation and active in more than 30 countries. The study covered 15 recommendations, with hundreds of clinical scenarios in total.

For each recommendation, the researchers paired cases in which the intervention was redundant with closely matched controls in which it was medically necessary. One example contrasted imaging for acute lower back pain after a person lifted furniture with new, nocturnal back pain in a woman with a history of breast cancer. Another contrasted antibiotics for a common viral cold with antibiotics for confirmed bacterial streptococcal throat infection.

Such pairs tested whether a model would respond to the clinical distinction rather than simply accepting or rejecting the intervention in every case. Six large language models were evaluated, producing thousands of model responses in total. These are aggregate results for the study rather than separate performance rates for every model.

The release therefore supports the narrower conclusion that the evaluated systems produced relatively few false-restraint errors and somewhat more over-testing errors overall. Results varied by clinical recommendation. For eight of the 15 Choosing Wisely recommendations, the models did not recommend a single unnecessary procedure in the responses evaluated for those scenarios.

The release does not describe these eight recommendation groups as clinically uncomplicated; it reports that performance was flawless for the unnecessary-intervention decision in those groups. The remaining recommendations accounted for the over-testing errors concentrated in the study. This pattern shows why the 5.4% aggregate rate should not be read as a uniform tendency across every clinical question.

The models often recognized low-value care, while their errors were concentrated in a smaller set of situations with less settled clinical boundaries. Routine electrocardiograms, or EKGs, in low-risk patients were one notable source of error. Models recommended an unnecessary EKG in 28.3% of responses, primarily when the case mentioned palpitations.

The report notes that an EKG can be a reasonable part of a workup in that context, making the distinction between an unnecessary test and a potentially justified one difficult to define. The study also found substantial differences among models on disputed cases. On the same clinical scenarios, one model recommended an unnecessary test in fewer than 10% of cases, whereas another did so in 80%, according to Guy Laban of Ben-Gurion University’s Department of Industrial Engineering and Management.

The contrast was reported as a striking disparity rather than a difference in overall accuracy across the full scenario set. Ultrasound for incidental thyroid nodules produced the largest identified category of over-testing. Models recommended unnecessary ultrasounds in 42.8% of responses involving small nodules discovered incidentally.

The clinical context is genuinely unsettled: the report says the American Thyroid Association and the American College of Radiology differ over whether such nodules should be imaged. The models’ behavior in this area therefore occurred where expert organizations themselves did not offer a single uniform practice. Dean Ariel, a family physician and researcher at Clalit Health Services in Tel Aviv District, said the language models appeared to have absorbed some ability to exercise restraint. “For the most part, they know when to stop, stumbling primarily where we doctors ourselves hesitate,” he said.

Laban said the findings also challenge the impression that a chatbot speaks with a single, unified “AI consensus.” When specialists debate whether an intervention is warranted, the models did not necessarily converge on one answer. Instead, their output depended on the clinical question and, in the reported comparisons, could vary dramatically among systems given identical cases. The study therefore presents an uneven picture of medical restraint: the models handled many scenarios in which unnecessary care was explicitly identified, but their recommendations became less consistent in areas marked by disagreement among clinicians.

The data also show that an overall percentage cannot describe the full behavior of an AI medical system without examining the particular recommendation and the responses to the scenarios built around it. The research also identified a separate communication challenge that the scenario-based evaluation did not test. Laban explained that telling a patient “you don’t need this test” is a delicate social act.

A human physician must stand firm in the face of an anxious or insistent patient, whereas language models may tend to go along with the person in front of them. The release describes this as an important next research question: whether an AI assistant will maintain correct clinical restraint when a patient pushes back. It does not report a separate experiment involving patient pressure, so the concern should not be treated as a measured result.

Overall, the study found that leading language models could often identify examples of low-value care, while also revealing sharp variation in ambiguous cases and the limits of evaluating medical AI through aggregate performance alone.

Subject of Research: Evaluation of large language model recommendations in Choosing Wisely clinical scenarios

Article Title: Does AI know when to say "no"? New study reveals AI chatbots recognize unnecessary medical tests, but stumble where human doctors disagree

Article References: Does AI know when to say "no"? New study reveals AI chatbots recognize unnecessary medical tests, but stumble where human doctors disagree. Original publication (Bibliographic details unavailable.)

Image Credits: AI Generated

DOI: Not provided

Keywords: large language models, medical AI, unnecessary testing, Choosing Wisely, clinical decision-making, over-testing, false restraint, thyroid nodules, electrocardiograms, patient safety, Does, know