The research team, led by Marwa Shembesh and Cortino Sukotjo and spanning institutions in the United States and abroad, assembled 300 multiple-choice questions drawn from the 2023 and 2024 examinations administered by the American College of Prosthodontists. Each year’s exam contained 150 questions, written by prosthodontics program directors and fellows of the college, and the correct answers were verified against the official answer key. Four large language models were put to the test: ChatGPT-3.5 and ChatGPT-4 from OpenAI, Microsoft’s Bing Copilot, and Google’s Gemini. The investigators created fresh accounts for each model to eliminate any influence of stored conversation history, used standardized zero-shot prompts instructing each system to pick the single best answer, and set the sampling temperature to zero for API-based queries to favor deterministic output.

The examination itself is structured around a three-tier hierarchy of topics that mirrors the written section of the American Board of Prosthodontics certification. Tier 1 covers the core of the specialty: complete dentures, fixed and removable partial dentures, dental biomaterials, occlusion, implant prosthodontics, and esthetics. Tier 2 spans adjacent disciplines such as periodontics, pharmacology, wound healing, and temporomandibular disorders, while Tier 3 collects the broader periphery, from biostatistics and diagnostic imaging to sleep disorders, oral pathology, and maxillofacial prosthodontics. In 2023, more than 61 percent of questions fell into Tier 1, with implant prosthodontics and removable partial dentures the most heavily represented topics. In 2024, dental materials and fixed prosthodontics dominated, and Tier 1 still accounted for nearly 57 percent of the exam.

The headline numbers are remarkable. In the 2023 examination, ChatGPT-4 answered 72.7 percent of questions correctly on the first testing round, edging out Gemini at 67.3 percent, Bing at 62.7 percent, and ChatGPT-3.5 at 62.0 percent. The average human resident that year scored 64 percent. In other words, the most advanced chatbot of its generation outperformed the typical prosthodontics resident on the same questions, at least descriptively. A year later the picture shifted: Gemini led the first round with 66.7 percent, followed by ChatGPT-4 at 64.0 percent, Bing at 60.0 percent, and ChatGPT-3.5 at 44.7 percent, against a resident average of 61 percent. Because the human and machine scores were not compared with paired statistical tests, the authors caution that these comparisons are descriptive rather than proof of superiority.

Statistical analysis told a more nuanced story. In 2023, the differences among the four models never reached statistical significance at either testing point. In 2024, however, the overall difference was significant at the first time point, with ChatGPT-4, Bing, and Gemini each significantly outperforming ChatGPT-3.5 after Bonferroni correction; Gemini’s advantage over the older model amounted to 22 percentage points. By the second round two weeks later, the omnibus difference was still significant, but no individual pairwise comparison survived correction, underscoring how unstable model rankings can be across sessions and examination years.

Perhaps the most unsettling finding concerns consistency. When the researchers repeated the entire examination two weeks after the baseline session, three of the four models held steady in 2023, with observed agreement between sessions ranging from 78.7 to 85.3 percent and Cohen’s kappa values between 0.518 and 0.647. But in 2024, ChatGPT-4’s accuracy dropped from 64.0 percent to 51.3 percent, a statistically significant decline of 12.7 percentage points. The authors note that unobserved changes on the provider side, such as backend model updates between testing sessions, cannot be excluded as a source of this variability, since exact platform-level technical details were not retained during data collection. For educators, the message is clear: the same chatbot asked the same question two weeks apart may give a different answer, even under tightly controlled conditions.

The topic-tier analysis added another layer of insight. Descriptively, all four models found Tier 3 questions, the broad peripheral subjects, easiest, pooling to 74.2 percent accuracy in 2023 and 69.9 percent in 2024, while Tier 1, the heart of prosthodontics, proved hardest at 61.1 percent and 53.1 percent respectively. After accounting for repeated responses to the same questions using generalized estimating equations, the tier effect was statistically significant only in 2024, when Tier 3 questions carried roughly twice the odds of a correct answer compared with Tier 1. Within the core Tier 1 material, ChatGPT-4 and Gemini both significantly outperformed ChatGPT-3.5, with odds ratios of 1.80 and 1.92 respectively, while Bing’s advantage did not survive statistical adjustment.

Counterintuitively, newer was not always better. In the 2023 exam, the older ChatGPT-3.5 actually beat ChatGPT-4 on Tier 2 questions at baseline, 77.8 percent to 70.4 percent, and on Tier 3 as well. In 2024, ChatGPT-3.5’s second-round total accuracy of 52.7 percent narrowly exceeded ChatGPT-4’s 51.3 percent. The authors point to similar reversals reported elsewhere, including cases where GPT-3.5 outperformed GPT-4 on classification tasks because its simpler pattern recognition avoided overgeneralization. Model performance, they conclude, depends on the interplay of architecture, training data, and question type rather than a simple hierarchy of model generations.

The study is the first to evaluate large language models on the National Prosthodontic Resident Examination, filling a gap in a literature that has mostly focused on medicine, periodontology, and endodontics. Its implications reach beyond dentistry. The findings suggest that while chatbots can serve as supplementary study aids for residents reviewing implant dentistry or dental materials, they cannot yet be trusted as authoritative sources, particularly on the specialized core knowledge that defines a specialty. The authors also flag concerns about examination integrity if candidates gain access to such tools during secure testing, and they emphasize that model-generated content must always be verified against reliable sources because these systems can fabricate plausible-sounding but incorrect information, a phenomenon known as hallucination.

The researchers acknowledge limitations that temper the conclusions. The question bank is proprietary and accessible only to members of the American College of Prosthodontists, the dataset consisted solely of multiple-choice items that may not capture the complexity of real clinical decision-making, and the two-week testing window cannot predict how performance might drift with future model updates. Future work, they suggest, should test open-ended and case-based questions that better mirror chairside judgment, extend the evaluation across longer periods and more specialties, and probe how training strategies shape performance across different question types. For now, the study stands as a vivid snapshot of a fast-moving frontier: machines that can nearly match the specialists of tomorrow on their own exam, yet still wobble from one week to the next, and that therefore belong beside the textbook, not in place of the clinician.

Subject of Research: Performance of large language models on the American College of Prosthodontists national resident examination

Article Title: The performance of different large language models in the national prosthodontic resident examination in 2023 and 2024

Article References: Shembesh, M., Koseoglu, M., Fang, Q., Gheisarifar, M., Kattadiyil, M. T., Yuan, J. C.-C., Barao, V. A., & Sukotjo, C. (2026). The performance of different large language models in the national prosthodontic resident examination in 2023 and 2024. Heliyon, 12(15), Article e45465. https://doi.org/10.1016/j.heliyon.2026.e45465

Image Credits: AI Generated

DOI: 10.1016/j.heliyon.2026.e45465

Keywords: artificial intelligence, large language models, ChatGPT, Google Gemini, prosthodontics, dental education, medical licensing exams, test-retest reliability, hallucination, American College of Prosthodontists, machine learning, healthcare education

The core of the argument rests on an uncomfortable observation about how medicine trains its practitioners. Medical education has long treated knowledge as the central currency of doctoring. Aspiring physicians spend years memorizing minutiae: the neuroanatomical pathways that link a left middle cerebral artery stroke to right-sided weakness and aphasia, the mechanisms by which Staphylococcus aureus evades beta-lactam antibiotics, and the serum thresholds at which lithium becomes acutely toxic. The system rewards immediate recall through full-day licensing and board certification examinations, as well as the ritual of rapid-fire questioning during clinical rounds. Yet fact retrieval, protocol recitation, and drug-interaction flagging are precisely the tasks that frontier AI systems now perform at or above expert level. Recent research cited in the correspondence shows that general-purpose large language models can outperform specialized clinical AI tools on medical benchmarks, a finding that underscores how quickly machine competence is spreading across the cognitive territory that medical training once monopolized.

Ng’s warning is stark in its implications: if medicine continues to define expertise primarily as the possession of specialized information, physicians may be training for a version of medicine that is going extinct. The literature on AI in medicine has focused heavily on what clinicians should know about AI, including its technical fundamentals, potential utility, and privacy and ethical risks. Far less attention has been paid to what clinicians should know about being clinicians when algorithms can do much of the thinking. That inversion, from machine literacy to human identity, frames the four clinical competencies the correspondence identifies as durable. Their expression may evolve alongside AI’s capabilities, but their underlying core, Ng argues, deserves continued emphasis in physician training.

The first competency is embodied presence, which Ng describes as a clinical tool in its own right. Studies have found that patients sometimes perceive AI-generated responses as more empathic than human ones, but empathy is only one facet of the physician’s relational engagement. Sitting with patients who are frightened about the future, ambivalent about their options, or genuinely difficult to like, physicians may notice their own countertransference: unexpected warmth, creeping anxiety, or even irritated boredom. These feelings are not distractions to be suppressed. They reveal how patients navigate the world, data that cannot easily be gleaned from the words in a chart. A physician’s embodied personhood, in this view, remains an instrument that guides decision-making, and reflective self-awareness about patient interactions should be valued as highly as diagnostic accuracy and pharmacology.

The second competency concerns participation, a capacity that detection alone cannot replace. AI-generated answers are bounded by the information provided to them. A standalone model can often produce the correct medical recommendation when all salient facts are entered, but research indicates that this accuracy decreases significantly when actual humans are involved, because patients frequently do not know which details are clinically relevant. Physicians therefore retain a critical role in eliciting and integrating the nuances of a patient’s narrative. Even as multimodal systems learn to analyze words, speech prosody, and facial expressions, and even if their capabilities eventually expand to encompass all five senses, Ng draws a sharp technical distinction: detection is different from participation. Diagnostic clarification and therapeutic work are co-created in the encounter, as physicians respond not only to what patients say but to how they say it, the hesitations, contradictions, posture shifts, and averted gazes, as well as to what remains unspoken.

The correspondence illustrates this point with vivid clinical vignettes. The patient who insists he is pursuing a healthier lifestyle while clutching a bag of fast food, or the adolescent who reports feeling close to her parents while sitting in the opposite corner of the room, present observations that shape interviews and treatment planning in ways not easily encoded into algorithms. These are not marginal details; they are often the hinge on which an accurate formulation turns. Human expertise, in its subtlety, still carries weight in the age of AI, precisely because the machine’s input is limited to what it is given, while the physician’s perception operates on the full, messy texture of the encounter as it unfolds in real time.

The third competency is discernment, which Ng predicts will matter more than information delivery as AI saturates the knowledge landscape. Patients will increasingly come to physicians not from a dearth of facts but from an overwhelm of them. Yet facts alone cannot explain to a family why the guideline-recommended treatment may still be the less ideal choice given their circumstances, nor can facts help a patient make peace with a decision when every option exacts a cost. Access to technically accurate information does not inherently produce effective decision-making. No matter how capable AI becomes at enumerating the full spectrum of risks, benefits, and alternatives, the physician is needed to help patients sift through the information, narrow the options, and arrive at decisions that are both medically and personally appropriate. In an era of infinite second opinions, the doctor becomes a filter and a guide rather than a source.

The fourth competency is the capacity to make decisions while acknowledging uncertainty, a skill Ng calls the psychiatrist’s native habitat. The pathophysiology of many mental illnesses remains incompletely understood, diagnostic boundaries can blur, and treatments are sometimes reached by trial and error. Medicine as a whole contains similar territory: answers that remain unknown, evidence limited by caveats, and professional consensus still evolving. The threshold between established knowledge and clinical intuition is not always clear, and learning to discern where that threshold lies, then helping patients navigate the ambiguous terrain beyond it, will remain essential. The contrast with machine behavior is pointed. AI chatbots may confidently hallucinate rather than admit not knowing, whereas good physicians acknowledge uncertainty and move forward toward a shared decision rather than attempting to conceal or prematurely resolve the unknown.

Ng’s conclusion is that AI literacy among clinicians is necessary but not sufficient. The practice of medicine should be built not only around what AI can do more efficiently but around what AI can never be: an embodied participant who helps patients navigate uncertainty and vulnerability. In short, the clinician’s humanity. These capacities are hard to measure and therefore easy to dismiss in an age of quantification, when board scores and benchmark accuracies dominate the conversation about medical competence. But they remain central to effective care, and the correspondence suggests that medical education should resist the gravitational pull toward training students as information processors whose output can be matched by a model. As information becomes ubiquitous and certitude can be cheerfully simulated by a chatbot, the relational skills from the psychiatrist’s toolkit, presence, participation, discernment, and comfort with ambiguity, may prove to be the most future-proof competencies in the entire medical curriculum.

The broader significance of the argument extends beyond psychiatry. Every specialty that touches human suffering, from oncology to geriatrics to primary care, will confront the same question as generative systems mature: when the machine can retrieve, reason, and recommend, what remains for the person in the white coat? Ng’s answer, drawn from a specialty that never depended on technology for its core work, is that the answer was never primarily in the facts. It was in the relationship. Whether medical educators act on that insight, and begin measuring and rewarding these relational capacities with the same rigor applied to factual recall, may determine whether the next generation of physicians enters a profession that still needs them, or one that has quietly redefined them out of existence.

Subject of Research: The enduring human competencies of physicians, particularly psychiatrists, in the age of generative artificial intelligence

Article Title: What Are Physicians For? How Psychiatry’s Competencies Endure Into the Age of AI

Article References: What Are Physicians For? How Psychiatry’s Competencies Endure Into the Age of AI. (n.d.). https://doi.org/10.1007/s40596-026-02448-z

Image Credits: AI Generated

DOI: 10.1007/s40596-026-02448-z

Keywords: generative AI, psychiatry, medical education, physician competencies, clinical empathy, countertransference, medical decision-making, clinical uncertainty, large language models, doctor-patient relationship, AI literacy, Mayo Clinic