Pragmatic evaluation can strengthen early medical education innovation

A 2026 editorial in the Journal of General Internal Medicine argues that educational innovations developed under real-world constraints can be evaluated pragmatically, with methods matched to their goals and stage of development. The editorial uses three studies in medical education to show how early evidence about feasibility, learning, uptake, and workforce development can guide stronger subsequent research. It presents pragmatic evaluation as compatible with rigorous scholarship rather than as a substitute for it.

The first example concerns “Just in Time Training for Internal Medicine Residents to Initiate State-Mandated Waiting Periods for Abortion Care.” The intervention combined brief, workflow-focused instruction on abortion law with an electronic medical record smart phrase checklist. It was intended to help internal medicine residents initiate the state-mandated waiting period for abortion care in authentic clinical settings. The evaluation paired embedded knowledge questions and structured participant feedback with iterative refinement across sessions, and it also examined electronic medical record smart phrase use as an exploratory signal of uptake. The study was an early-stage educational evaluation in a clinical workflow, not evidence that the training improved patient outcomes.

The editorial describes the layered design as aligned with the intervention’s aim of workflow adoption rather than knowledge acquisition alone. The measures captured feasibility, immediate learning, iterative improvement, and initial implementation, although the reported outcomes remained early and indirect. The authors frame these measures as steps toward outcomes that matter to patients, while emphasizing that such a connection requires further evaluation. The example illustrates how process and uptake measures can be useful when an innovation is being introduced into a real clinical environment.

The second example, “Building an Obesity Medicine Workforce: Implementation of a Comprehensive Resident Training Pathway in an Underserved Care Environment,” addresses a longitudinal obesity medicine pathway for internal medicine residents. The pathway included asynchronous learning, supervised continuity clinic experiences, and professional development intended to support preparation for the obesity medicine board examination. Because the program was evaluated in a small residency cohort, the authors used pre- and post-surveys to assess perceived confidence, practice patterns, and interest in future obesity medicine certification. These self-reported measures were selected for a workforce-development program whose immediate purpose was to prepare residents for future roles.

According to the editorial, perceived preparedness and commitment to future certification are plausible markers along the path from education to future practice. They do not establish patient-level impact, and the study cannot determine whether residents subsequently changed practice or achieved certification. The example therefore shows both the usefulness and the limits of pragmatic assessment: self-reported measures can provide contextually appropriate early evidence for an emerging pathway, but they require later links to observed behavior, sustained implementation, or patient-level outcomes.

The third example is “Alert and Oriented: Translating Vignette-Based Reasoning Pedagogy into an Open Access Podcast for Early Learners.” The authors created a student-led, vignette-based clinical reasoning podcast for use in clinical clerkships. The format adapted an established reasoning pedagogy to an open-access digital resource intended to broaden access to clinical reasoning instruction for early learners. Rather than beginning with downstream learning outcomes, the evaluation examined listener analytics, including audience reach, downloads, and engagement, as early indicators of educational uptake.

For an open-access educational tool, the editorial says, reach and uptake are foundational indicators of whether the innovation is being used and engaged with. Analytics and other trace data can therefore be pragmatic evaluation tools in dissemination-focused work. At the same time, the podcast study points to a next step for digital educational media: connecting engagement metrics with learning, practice change, or other downstream outcomes. The available evidence described in the editorial supports the evaluation of use, not the claim that podcast engagement changes clinical learning or practice.

Across the three examples, the editorial identifies a recurring gap in educational innovation research: data collection methods are often described more clearly than the evaluation approach or conceptual logic guiding them. Surveys, feedback forms, analytics, and usage data show how information was gathered, but may not explain why particular outcomes were selected, how they align with program goals, or what conclusions they can support. The authors caution that pragmatism should not mean atheoretical or unplanned evaluation. Conceptual anchors such as feasibility and implementation constructs, developmental program evaluation, Miller’s pyramid, and Glassick’s criteria can help authors situate innovations, justify outcome choices, and clarify the evaluation stage.

The editorial recommends that scholars and educators state the primary educational goal, select outcomes matched to that goal and the innovation’s development stage, identify the conceptual approach guiding interpretation, and specify what the evidence can and cannot support. Prospective planning for the next phase can include embedding surveys in program delivery, extending follow-up, pooling data across cohorts or institutions, adding observed or intermediate behavioral outcomes, or linking digital and electronic workflow metrics to learner- or practice-level changes. Journals, editors, and reviewers also have a role: innovation studies documenting feasibility, uptake, or proof of concept need not meet the expectations of mature later-phase research, but they should still demonstrate evaluative forethought. In this developmental view, early educational signals can become stronger evidence through intentional planning while ultimately supporting better clinical practice and patient care.

Subject of Research: Pragmatic evaluation of early-stage medical education innovations

Article Title: Pragmatic Evaluation in Educational Innovation: Valuing Early Evidence, Strengthening What Comes Next

Article References: Qua, K., & Windish, D. M. (2026). Pragmatic Evaluation in Educational Innovation: Valuing Early Evidence, Strengthening What Comes Next. Journal of General Internal Medicine. https://doi.org/10.1007/s11606-026-10815-7

Image Credits: AI Generated

DOI: 10.1007/s11606-026-10815-7

Keywords: medical education, educational innovation, pragmatic evaluation, implementation research, program evaluation, clinical training, obesity medicine, abortion care, clinical reasoning podcast, workforce development, Pragmatic, Evaluation