Cohen standard setting closely matches Angoff in medical school exams

A retrospective analysis of 44 multiple-choice examinations found that the Cohen standard-setting method produced passing scores and student pass rates broadly comparable to the Angoff method across four academic years of an undergraduate medical doctor program. Mean cutoffs were 63.13 under Angoff and 64.16 under Cohen, while mean pass rates were 75.45% and 74.69%, respectively. The study assessed whether the simpler Cohen approach could offer a lower-burden alternative or a monitoring benchmark without materially changing high-stakes assessment decisions.

The exam-level study included three high-stakes phase-progression examinations, eight National Board of Medical Examiners Subject Examinations and 33 course final examinations. Angoff cutoffs for locally developed exams were established through a modified, faculty-based procedure in which faculty estimated the performance of a minimally competent candidate. Publisher-recommended Angoff cutoffs were used for the NBME examinations. In contrast, the Cohen cutoff was calculated locally as 75% of the 90th-percentile score, with 75% also serving as the institution’s official MD program pass mark. The researchers paired four-year average cutoffs and pass rates for the two methods.

Agreement between the methods was high when all examinations were considered together. The mean paired difference in cutoff was 1.03 points in favor of Cohen, with a 95% confidence interval of 0.45 to 1.60, and the mean absolute difference was 1.76 points. Pearson correlation between exam-level cutoffs was 0.96, and the absolute-agreement intraclass correlation coefficient was 0.951. Cutoff differences were no more than 2 points in 28 of 44 exams, or 63.6%; no more than 3 points in 36 exams, or 81.8%; and no more than 5 points in every exam.

Pass rates showed similarly strong overall agreement. Cohen generated a mean pass rate 0.76 percentage points below Angoff, with a 95% confidence interval of -1.99 to 0.47 percentage points, while the mean absolute difference was 3.23 percentage points. The exam-level correlation was 0.95 and the absolute-agreement intraclass correlation coefficient was 0.948. Pass-rate differences were within 5 percentage points for 35 examinations, or 79.5%, and within 10 percentage points for all 44 examinations. Exploratory paired t-tests were conducted at the exam level.

The combined results were not uniform across assessment types. Agreement was weaker for NBME pass rates, where the intraclass correlation coefficient was 0.451, and for course-final cutoffs, where it was 0.682. These category-specific findings qualify the overall conclusion: Cohen performed similarly across the full assessment dataset, but its agreement with Angoff varied depending on the kind of examination. The results therefore support considering Cohen as a lower-burden option or as a monitoring benchmark, rather than establishing that the two methods are interchangeable in every assessment context.

Subject of Research: Comparison of Cohen and Angoff standard-setting methods for multiple-choice assessments in a six-year undergraduate medical doctor program

Article Title: Cohen standard setting yields cutoffs and pass rates comparable to Angoff: a retrospective comparison of MCQ assessments in an MD program

Article References: Shaban, S., Zafar, I., & Magzoub, M. E. (2026). Cohen standard setting yields cutoffs and pass rates comparable to Angoff: a retrospective comparison of MCQ assessments in an MD program. BMC Medical Education. https://doi.org/10.1186/s12909-026-10414-4

Image Credits: AI Generated

DOI: 10.1186/s12909-026-10414-4

Keywords: medical education, standard setting, Cohen method, Angoff method, multiple-choice exams, cutoff scores, pass rates, assessment, medical doctor program, NBME examinations, Cohen, standard