Clinical diagnosis is inherently iterative. Physicians gather information from patient histories, physical examinations, and diagnostic tests, revising their differential diagnoses as new evidence emerges and judging when sufficient certainty exists to conclude the workup or begin treatment. Although language models have shown impressive medical reasoning on written examinations and structured clinical cases, conventional evaluations typically hand the model complete case information upfront. According to the authors, compressing diagnosis into a single inference step means static benchmarks cannot measure whether a system asks the right questions, refrains from indiscriminate testing, updates its differential diagnosis as evidence accrues, or recognizes when enough certainty has been achieved to stop ordering tests. This limitation, they argue, can overstate diagnostic competence.

To close that gap, the team created the Sequential Diagnosis Benchmark (SDBench), which transforms 629 clinical cases from The New England Journal of Medicine and emergency department encounters into interactive, step-by-step diagnostic challenges. Using SDBench, the researchers evaluated DxO, an AI orchestrator powered by GPT-5 that guides diagnostic reasoning by selecting questions and tests with cost considerations in mind. Across the evaluated cost-accuracy range, DxO established a more favorable balance between diagnostic accuracy and testing costs than the participating physicians. Depending on the budget constraints applied, the system achieved accuracies of 79.6 percent to 84.5 percent, with average costs ranging from 1,303 dollars to 8,945 dollars per case.

The comparison with human performance was stark. Physicians assessing curated clinical conference cases achieved average diagnostic accuracies of 22.3 percent without reference tools and 36.1 percent with them. Even the highest-performing physician, whose accuracy reached 51.8 percent, was surpassed by DxO in both accuracy and cost. The AI system also followed a distinctly different diagnostic strategy: it gathered substantially more patient-history information before ordering its first test than the physicians typically did, a pattern consistent with its design goal of avoiding unnecessary and expensive testing. On routine emergency department cases, DxO achieved 80.3 percent accuracy at an average cost of 4,980 dollars per case when no budget constraints were applied.

The authors caution that these findings, while suggesting that AI orchestration could support more efficient diagnosis, do not yet establish effectiveness in real-world clinical practice or demonstrate an impact on patient outcomes; further evaluation is needed. They also stress a fundamental limitation of the current system: DxO cannot perform physical exams or observe patients directly, which means it operates on information that must be gathered and relayed by human clinicians. Within those limits, the researchers say the system shows promise as a supplement to a physician’s workflow rather than a replacement for bedside assessment. The study, titled “Sequential diagnosis with language models,” positions sequential, cost-aware benchmarking as a more demanding and arguably more realistic test of medical AI than static question-answering evaluations, measuring not just whether a model can reach the right answer with full information, but whether it can reach it the way a careful clinician would, one question and one test at a time.

Subject of Research: Sequential AI diagnostic reasoning benchmarked against physicians on accuracy and testing costs

Article Title: AI system boosts diagnostic accuracy while reducing medical testing costs

Article References: AI system boosts diagnostic accuracy while reducing medical testing costs. Original publication (Bibliographic details unavailable.)

Image Credits: AI Generated

DOI: Not provided

Keywords: artificial intelligence, diagnostic reasoning, language models, SDBench, DxO, GPT-5, clinical diagnosis, testing costs, benchmark, physician comparison, Science journal, emergency medicine