Teaching the Teachers: Pilot Study Puts Surgeons’ Feedback Skills to the Test

Feedback is the engine of medical training, yet the physicians expected to deliver it often receive almost no instruction in how to do so. A new pilot study published in Global Surgical Education, the journal of the Association for Surgical Education, set out to change that by testing a structured teaching exercise designed to sharpen two skills that every clinical supervisor needs: giving effective feedback and scoring a learner’s procedural performance accurately. The results, published by a team led by Binh Le of the University of Nebraska Medical Center, offer a candid and somewhat surprising look at what happens when faculty are asked to practice what they preach.

The tool at the center of the study is the objective structured teaching exercise, or OSTE, a close cousin of the objective structured clinical examination that has long been used to test medical students. In an OSTE, teachers step into the learner’s seat. They encounter a simulated learner, an actor or colleague trained to portray a student with a specific educational problem, and must demonstrate their teaching and feedback techniques in a controlled, observable setting. A trained assessor scores their performance against defined criteria. The approach has been used in pharmacy, internal medicine, and general clinical education, but the Nebraska team wanted to know whether it could work in a distinctly procedural environment, where feedback must address hands-on technical skill rather than bedside conversation alone.

For their pilot, the researchers recruited six physicians who supervise central line placements at a single academic institution. Central venous catheter insertion is a high-stakes procedure: it demands sterile technique, precise anatomical knowledge, and flawless execution, and errors can lead to serious complications. Supervisors who watch trainees perform this procedure must both judge the performance against a rubric and communicate their observations in a way that improves the learner rather than demoralizes them. That dual burden, assessment plus feedback, made the central line setting an ideal proving ground for the OSTE format.

Each participant completed three sessions built around a single case. In every session, the physician watched a video of a simulated central line placement, graded the procedure using a standardized rubric, and then entered a room with a simulated learner to deliver feedback on the performance. Two independent judges evaluated the feedback each time: the simulated learner, who experienced the feedback firsthand, and a master assessor trained to score teaching performance. This dual-evaluation design allowed the researchers to capture both the subjective reception of the feedback and a more objective measure of its quality.

Between the first and second sessions, participants completed a training module covering two things: how to use the scoring rubric optimally, and how to apply the Pendleton Method of feedback. The Pendleton Method, a structured feedback framework well known in clinical education, asks the feedback giver to first identify what the learner did well, then what could be improved, and to build the conversation around the learner’s own analysis before adding the supervisor’s perspective. The researchers also collected qualitative data, interviewing participants and administering a survey at the end of the third session, then analyzing responses thematically. Quantitative scores from before and after the training were compared using PC SAS version 9.4.

The feasibility findings were unambiguous. All six participants found the OSTE feasible for practicing feedback in a procedural setting, and all found it helpful for enhancing their feedback skills. For a study population of busy supervising physicians, a group notoriously difficult to engage in faculty development, that endorsement matters. The exercise was deliberately compact, requiring only a video, a rubric, a simulated learner, and a short debriefing, which the authors suggest makes it a flexible option for institutions that cannot spare faculty for lengthy workshops.

The performance data told a more complicated story. When the researchers compared feedback skill scores before and after the training module, they found no statistically significant improvement. The simulated learners’ ratings showed a p-value of 0.125, the master assessor’s ratings a p-value of 0.313, and the combined ratings a p-value of 0.875, none reaching the conventional threshold for significance. In other words, a single training module on rubric use and the Pendleton Method did not measurably change how these experienced physicians delivered feedback, at least as captured by this instrument and this sample size.

Even more striking was what happened to scoring accuracy. After training, participants’ grading of the procedural video became significantly less accurate, with p-values of 0.031 for both the case score and the total score. The authors interpret this counterintuitive result cautiously, but it echoes a known phenomenon in assessment research: rater training can shift how raters distribute their judgments, sometimes making them harsher or more internally consistent without making them more accurate. It also connects to the well-documented Dunning-Kruger effect, cited in the study’s bibliography, in which people with limited skill in a domain tend to overestimate their own competence. Physicians who believed the training helped them may nonetheless have recalibrated their scoring in ways that moved them away from the reference standard.

The study’s limitations are those of any pilot. Six participants at a single institution is a small sample, and with numbers that small, statistical power is limited; a real effect could be masked, or a spurious one amplified. The intervention itself was brief, a single module between sessions, and the authors acknowledge that the most effective OSTE design for procedural settings remains an open question. They call for further study to refine the format, suggesting that more intensive or repeated training, different feedback frameworks, or larger and more diverse cohorts might produce measurable gains where this lightweight version did not.

Still, the study makes a genuine contribution by dragging faculty development into the procedural arena. Most OSTE research has focused on communication-heavy teaching, and the systematic review literature cited by the authors confirms that the format is widely used but unevenly studied. By adapting it to central line supervision, complete with rubric-based scoring of a technical performance, the Nebraska team has demonstrated that the machinery works in this setting, even if the training content needs refinement. The broader lesson for medical education may be that engagement and improvement are separate hurdles: getting experienced physicians to practice feedback in a safe simulated environment is achievable, but changing ingrained feedback habits and calibrating assessment judgment appears to demand more than a single module. As competency-based medical education spreads and workplace assessment becomes central to certification, the question of how to train the assessors themselves will only grow more urgent. This pilot study, honest about its null and negative findings, marks a realistic first step toward answering it.

Subject of Research: Objective structured teaching exercises for training physician feedback delivery and procedural scoring accuracy in medical education

Article Title: Leveraging objective structured teaching exercises (OSTE) to enhance feedback delivery skills and scoring accuracy in medical education: a pilot study

Article References: Le, B., Ehsan, L., Anyagwa, O. E., Ehsan, E., Buesing, K., Shostrom, V., Rodrigues-Armijo, P., Krause, C., Lipman, J., & Sheppard, O. O. (2026). Leveraging objective structured teaching exercises (OSTE) to enhance feedback delivery skills and scoring accuracy in medical education: a pilot study. Global Surgical Education – Journal of the Association for Surgical Education, 5(1), Article 134. https://doi.org/10.1007/s44186-026-00535-1

Image Credits: AI Generated

DOI: 10.1007/s44186-026-00535-1

Keywords: OSTE, medical education, feedback delivery, simulated learner, central line placement, Pendleton Method, faculty development, scoring accuracy, surgical education, pilot study, assessment rubric, simulation