The study, published in Global Surgical Education, the Journal of the Association for Surgical Education, analyzed data from eight consecutive annual administrations of the assessment between 2018 and 2025. In total, the researchers examined results from 5,869 postgraduate year one surgery residents drawn from 626 cohorts across 224 unique training programs. The scale of the dataset matters: this is not a survey of a handful of programs or a single academic year, but a sustained national measurement of decision-making readiness at the precise moment residents enter surgical training. Because the assessment is formative rather than punitive, programs receive feedback intended to guide supervision and curriculum design rather than to penalize individuals.

The instrument itself is built on a well-established psychometric framework called the key-features approach, originally developed for the Medical Council of Canada’s licensing examinations. Rather than testing broad medical knowledge, key-feature problems zero in on the specific decision points in a clinical scenario where expertise matters most and where errors are most likely to cause harm. The ACS ERRA presents residents with 40 case-based clinical scenarios spanning 20 clinical topic areas, encompassing 140 discrete decision points. Each case describes a realistic early-training situation, such as a postoperative patient with new symptoms, and asks the resident to choose the correct next action at the moments that critically shape the patient’s trajectory. This design deliberately targets the kind of judgment that separates a safe beginner from a dangerous one.

Across the full eight-year window, overall performance was strikingly stable. The mean score across all residents was 65 percent correct, with a standard deviation of 8 percent, and multi-year regression analyses found no meaningful drift in the aggregate results over time. On its face, that consistency suggests that the general level of clinical decision-making skill among entering surgery residents has not changed appreciably across nearly a decade, despite evolving medical school curricula, the disruptions of the COVID-19 pandemic on clinical rotations, and ongoing national debates about the transition to residency. Whether that stability represents a plateau of adequate preparation or a persistent systemic shortfall is precisely the kind of question the assessment was designed to illuminate.

The more revealing story lies beneath the average. Performance varied substantially across clinical domains, and the pattern of variation is clinically meaningful. Residents scored lowest on scenarios involving a change in respiratory rate, where the mean score was just 51 percent, followed by abdominal pain at 57 percent and irregular heartbeat at 58 percent. By contrast, they performed considerably better on scenarios involving somnolence, hypotension, and chest pain, each with mean scores of 73 percent. The asymmetry is instructive. Chest pain and hypotension are dramatic, high-salience presentations that receive heavy emphasis in medical education, while a subtle shift in respiratory rate or an unexplained abdominal complaint may demand more nuanced pattern recognition, the kind of contextual judgment that new graduates have had fewer opportunities to practice.

The longitudinal analysis added a second layer of concern. Mean scores in four specific domains, including change in respiratory rate, traumatic extremity ischemia, oliguria, and fever etiologies, declined significantly during the later years of the study period, with decreases ranging from 3 to 8 percent and all reaching statistical significance at p less than 0.05. A declining trend in recognizing traumatic extremity ischemia, a time-critical vascular emergency in which delayed diagnosis can cost a limb, is particularly noteworthy for surgical educators. The authors caution that the assessment is formative and the declines are domain-specific rather than global, but the trends suggest that certain categories of clinical reasoning may be eroding in the preparation of entering residents, warranting attention from medical schools and residency programs alike.

Perhaps the most patient-safety-relevant metric in the study is the frequency of potentially harmful actions, or PHAs, which are answer choices that would actively endanger the patient rather than merely reflect suboptimal judgment. Potentially harmful actions were most frequently identified in hypoxemia scenarios, where 46.3 percent of residents selected a harmful option, followed by irregular heartbeat at 33.8 percent, abdominal pain at 32.0 percent, and fever-related scenarios at 30.6 percent. In other words, in some of the most common situations a new resident will encounter, roughly one in three chose a course of action that could genuinely harm a patient. These are not obscure corner cases; abdominal pain, fever, and abnormal heart rhythms are bread-and-butter clinical presentations on any surgical ward.

The authors, led by Kathleen R. Liscum of the Division of Education at the American College of Surgeons, together with Adnan A. Alseidi, Yoon Soo Park, and colleagues, frame these findings not as an indictment of new residents but as an argument for precision in early training. Surgical education has increasingly embraced the idea that readiness cannot be assumed from medical school grades or licensing examination scores alone, and that targeted, objective data collected at entry allows programs to calibrate supervision to actual need. The study’s conclusions emphasize that the substantial variability in decision-making readiness, combined with frequent potentially harmful actions in specific domains, demonstrates that early assessment can help programs identify both individual residents and clinical topic areas that may benefit from closer supervision and early intervention at the very beginning of surgical training.

The broader context reinforces why this matters. Research on the so-called July effect has long examined whether the annual influx of new residents affects patient outcomes, with systematic reviews yielding mixed but persistent concerns. Diagnostic errors remain a leading source of preventable harm in medicine, and studies of diagnostic error consistently implicate failures of clinical reasoning rather than gaps in factual knowledge. Meanwhile, the transition from medical student to resident has been repeatedly identified as a period of heightened vulnerability for both learners and patients, prompting national efforts such as the Association of American Medical Colleges’ Core Entrustable Professional Activities for entering residency. The ACS ERRA findings provide exactly the kind of granular, longitudinal evidence that such reform efforts have lacked.

For surgical programs, the practical implications are concrete. A program director reviewing ACS ERRA results can see not only how an individual incoming resident performed overall but which of the 20 clinical topic areas produced errors and whether those errors were merely suboptimal or potentially harmful. A resident who struggled with hypoxemia scenarios, for example, might receive focused simulation-based training and closer attending oversight during the first weeks of call, while a cohort-wide weakness in recognizing respiratory deterioration might prompt a revision of the program’s orientation curriculum. Because the assessment is repeated annually and now characterized across eight years, programs can also benchmark their cohorts against national data and track whether interventions actually move the needle. As formative assessment increasingly becomes a fixture of the transition to residency, this eight-year national dataset establishes a benchmark for entering surgical decision-making and a template for how the profession can measure, and ultimately improve, the readiness of its newest members.

Subject of Research: Clinical decision-making readiness of entering surgery residents measured by a national formative assessment

Article Title: Critical decision-making skills of entering surgery residents: eight years of formative assessment results from the American college of surgeons entering resident readiness assessment (ACS ERRA)

Article References: Liscum, K. R., Alseidi, A. A., Park, Y. S., Wasielewski, K., Blair, P. G., Terhune, K., & Sachdeva, A. K. (2026). Critical decision-making skills of entering surgery residents: eight years of formative assessment results from the American college of surgeons entering resident readiness assessment (ACS ERRA). Global Surgical Education – Journal of the Association for Surgical Education, 5(1), Article 168. https://doi.org/10.1007/s44186-026-00574-8

Image Credits: AI Generated

DOI: 10.1007/s44186-026-00574-8

Keywords: surgical education, clinical decision-making, resident readiness, formative assessment, transition to residency, patient safety, ACS ERRA, key-features approach, potentially harmful actions, surgery residents, psychometrics, precision surgical education