USMLE Step 2 Scores and Red Flags Track Emergency Medicine Resident Performance

Emergency medicine residency programs have spent recent years searching for better ways to choose among thousands of applicants, and a new retrospective analysis offers an exploratory, data-driven look at which screening measures actually track with performance on the job. The study, published as an open-access research article in BMC Medical Education, evaluated 133 residents who graduated from a single program between 2018 and 2024. The research team applied a multidimensional framework built around a specific hypothesis: that metrics capturing applied cognitive competence, such as licensing examination scores, would predict success differently from measures intended to capture non-cognitive professionalism, such as standardized letters and interview ratings. The answer, in this cohort at least, was more nuanced than the framework anticipated.

The researchers assembled a broad set of applicant data, including United States Medical Licensing Examination scores, core clerkship grades, Standardized Letters of Evaluation, interview scores, application red flags, and demographic variables. Resident performance was then rated retrospectively on a five-point Likert scale by core program leadership and Clinical Competency Committee faculty. The primary outcome dichotomized residents into a high-performer tier, defined as a score of 3 or 4, and a non-high tier, defined as 0 to 2, based on evaluations by four core faculty members. Ninety residents, or 67.7 percent of the cohort, landed in the high-performer category. The team analyzed the outcome with univariable logistic regressions and corroborated the findings with a proportional odds ordinal logistic regression sensitivity analysis.

Two application factors emerged with statistically significant positive associations. Higher USMLE Step 2 scores were linked to high performance, with an odds ratio of 1.40 per 10-point increase in the univariable model, a 95 percent confidence interval of 1.04 to 1.93, and a p-value of 0.03. Younger age at entry was also associated with high performance, with an odds ratio of 0.84 per year of age, a 95 percent confidence interval of 0.73 to 0.96, and a p-value of 0.02. Both associations remained consistent when the researchers reran the analysis using the ordinal model, which preserves the full ranking of performance ratings rather than collapsing them into two tiers. In that sensitivity analysis, the Step 2 odds ratio was 1.45 with a p-value of 0.01, and the age odds ratio was 0.88 with a p-value of 0.03.

The most striking result concerned application red flags. The data demonstrated complete separation: all three residents in the cohort who had red flags in their applications fell into the lower performance tier, yielding a Fisher’s exact p-value of 0.03. Because none of the red-flag applicants reached the high-performer group, the logistic model could not estimate a stable odds ratio for this factor, which is why the researchers reported the Fisher’s exact test instead. The authors caution, however, that with only three affected residents, this finding is exploratory and should not be treated as a definitive selection criterion.

Several measures that programs have long relied upon failed to differentiate performers in this cohort. USMLE Step 1 scores, core clerkship grades, interview scores, degree type, sex, and honor society membership showed no significant association with performance. A higher composite SLOE score actually had an unadjusted negative association with the primary binary outcome, with an odds ratio of 0.80, a 95 percent confidence interval of 0.64 to 0.99, and a p-value of 0.05. That counterintuitive signal was attenuated to non-significance in the ordinal model, where the odds ratio was 0.85 with a 95 percent confidence interval of 0.71 to 1.02 and a p-value of 0.07, suggesting the binary result may have been an artifact of how the outcome was dichotomized rather than a genuine predictive relationship.

The authors are careful to frame these findings as exploratory baseline insights for holistic review rather than definitive selection criteria. The study design carries important limitations: performance ratings were retrospective and subjective global assessments by program leadership, and the univariable comparisons were unadjusted for confounding variables. The single-center design also means the cohort reflects one program’s applicant pool and evaluation culture, so the results may not generalize to other institutions. The authors note that the landscape of emergency medicine residency recruitment has changed significantly in recent years, which is precisely why they argue that data-driven selection approaches are needed.

For admissions committees, the practical takeaway is a modest one. In this cohort, higher USMLE Step 2 scores demonstrated a positive association with subjective global performance ratings, while candidate red flags were associated with lower ratings. Conversely, historical composite SLOE scores and interview scores did not reliably differentiate higher global ratings. Given the limitations of retrospective subjective ratings and unadjusted comparisons, the findings provide a starting point for programs seeking to clarify why certain screening measures predict success, rather than a formula for replacing holistic review with a checklist of examination scores and demographic thresholds.

Subject of Research: Association between residency application factors and emergency medicine resident performance ratings

Article Title: Factors from residency applications associated with performance in emergency medicine: a retrospective study

Article References: Rafaeli, B. O., Huebinger, R. M., Van Meter, M. W., Stevens, G. S., Kosoko, A. A., & Cooper, B. L. (2026). Factors from residency applications associated with performance in emergency medicine: a retrospective study. BMC Medical Education. https://doi.org/10.1186/s12909-026-10672-2

Image Credits: AI Generated

DOI: 10.1186/s12909-026-10672-2

Keywords: emergency medicine, residency selection, USMLE Step 2, SLOE, medical education, holistic review, resident performance, retrospective study, Factors, residency, applications, associated