The study, published in BMC Nursing, was led by Xiaobei Ma and Xueyan Fan, who contributed equally, together with colleagues including Fengli Gao, Xin Liu, Yi Duan, and corresponding author Zhifeng Gao. Rather than relying on a generic nursing checklist, the team constructed a framework tailored specifically to the role of the nurse anesthetist, a profession in which errors in airway management, drug dosing, or vigilance during the post-anesthesia care phase can have immediate and serious consequences. The result is a hierarchical instrument comprising five domains, twenty second-level indicators, and one hundred individual scored elements, each anchored to explicit behavioral criteria.

Building such an instrument required a multi-stage methodology that blends structured expert consensus with quantitative weighting. The researchers began with a literature review and semi-structured interviews with experts, then ran a two-round Delphi consultation, a technique in which a panel of specialists independently rates proposed items and receives feedback between rounds until agreement stabilizes. To determine how much weight each domain should carry in the final score, they applied the analytic hierarchy process, a mathematical method that converts pairwise expert judgments into a consistent set of weights, checking that the judgments did not contradict one another through a consistency ratio threshold.

The preliminary measurement evaluation focused on content validity and reliability. Experts rated the relevance of each item, yielding a scale-level content validity index average of 0.95, very close to the perfect score of 1.0, while item-level modified kappa values, which correct expert agreement for chance, ranged from 0.831 to 1.000. To estimate inter-rater reliability, five nurses were independently scored by two prespecified evaluators at program entry; the total-score intraclass correlation coefficient using the two-way mixed-effects absolute-agreement single-measure definition, ICC(A,1), was 0.889, with a 95 percent confidence interval of 0.388 to 0.988. The authors are candid that the narrow confidence interval’s lower bound reflects the tiny reliability subsample, making this preliminary, low-precision evidence rather than comprehensive validation.

The longitudinal component is where the study becomes genuinely distinctive. The team retrospectively analyzed a census of all eligible nurses entering the training program between August 2016 and August 2025, defining T0 as program entry after pre-employment preparation and T1 through T6 as the completion of each of the first six months. Of 77 screened records, 68, or 88.3 percent, had complete T0 through T6 data and entered the analysis; five nurses resigned during the period and four had at least two consecutive missing monthly assessments. All analyses were performed in R version 4.5.2, with repeated-measures analysis of variance serving as the primary within-evaluator test of change over time.

The trajectory of improvement was striking. Mean self-assessment scores climbed from 35.95 at entry to 96.83 at month six, while faculty ratings rose from 29.20 to 95.32. After applying the Greenhouse-Geisser correction, which guards against violations of the statistical assumption of sphericity in repeated-measures designs, the time effects were highly significant for both perspectives, with P values below 0.001 and partial eta-squared values of 0.977 for self-ratings and 0.984 for faculty ratings, meaning that time explained nearly all of the variance in scores. In practical terms, new nurse anesthetists moved from roughly a third of the maximum score to near-ceiling performance within six months under this framework.

Perhaps the most psychologically interesting finding concerns the gap between self-perception and external judgment. At every single time point, self-ratings exceeded faculty ratings, but the size of that gap changed systematically. A participant-level mixed-effects model, which accounts for repeated observations nested within individuals, estimated the model-based difference between evaluator trajectories as largest at month two, at 9.51 points, and shrinking to just 1.50 points by month six. The time-by-evaluator interaction was significant in a likelihood ratio test yielding a chi-square statistic of 110.67 with six degrees of freedom and P below 0.001. The pattern suggests that new nurses begin with a degree of overconfidence relative to their supervisors’ assessments, then calibrate their self-appraisal as real clinical experience accumulates.

The framework also showed meaningful concurrent associations with established assessment tools. At month three, self-ratings correlated with written examination scores at a Spearman coefficient of 0.815 and faculty ratings at 0.582, while the corresponding correlations with objective structured clinical examination scores, the standardized practical tests widely used in medical education, were 0.580 and 0.558. By month six these associations remained positive, at 0.752 and 0.532 for the written examination and 0.689 and 0.340 for the OSCE. Notably, at program entry the correlations with the written examination were negligible, which makes sense: before any anesthesia-specific training, general knowledge tests say little about how a nurse will perform in the operating room, so the framework’s scores capture something the written test does not.

The authors also report an exploratory analysis in a separate prespecified five-person subset examining whether every individual improved from T0 to T6. All five showed positive change, but an exact two-sided sign test produced a P value of 0.0625, just shy of the conventional 0.05 threshold, a mathematical inevitability when only five participants are tested, since the smallest achievable two-sided P value for a sign test with five observations is exactly 0.0625. The team treats this honestly as suggestive rather than confirmatory, and the supplementary materials go further, documenting acceptability questionnaires completed by all 68 respondents, sensitivity analyses, and exploratory safety surveillance, an unusually transparent level of statistical disclosure for a single-centre education study.

What the study does not claim is perhaps as important as what it does. The authors explicitly state that their results provide preliminary, low-precision measurement evidence rather than comprehensive validation or causal proof that the training program itself drove the score increases, and they call for prospective multicenter evaluation before the framework is used for any high-stakes decisions such as hiring, promotion, or certification. Still, the implications are considerable. With anesthesia teams worldwide facing persistent staffing pressures, a validated, behaviorally anchored, month-by-month competency map could help educators target support precisely where new nurses lag, and the finding that self-assessment converges with faculty judgment over time offers a measurable signal of professional maturation. If larger multicenter studies replicate these trajectories, the humble scoring rubric developed in a Beijing hospital could become a template for how surgical specialties everywhere turn the anxious first months of a new clinician’s career into a quantified, coachable journey.

Subject of Research: Development and longitudinal evaluation of a competency assessment framework for newly recruited nurse anesthetists

Article Title: Development, preliminary measurement evaluation, and longitudinal application of a competency assessment framework for newly recruited nurse anesthetists

Article References: Ma, X., Fan, X., Gao, F., Liu, X., Duan, Y., & Gao, Z. (2026). Development, preliminary measurement evaluation, and longitudinal application of a competency assessment framework for newly recruited nurse anesthetists. BMC Nursing. https://doi.org/10.1186/s12912-026-05424-y

Image Credits: AI Generated

DOI: 10.1186/s12912-026-05424-y

Keywords: nurse anesthetists, competency framework, Delphi method, longitudinal assessment, mixed-effects model, content validity, inter-rater reliability, OSCE, nursing education, self-assessment, analytic hierarchy process, BMC Nursing

The examination emerged from a growing body of educational theory that distinguishes between assessment of learning, which measures what a candidate knows at a fixed moment, and assessment for learning, which uses evaluation itself as a developmental tool. Traditional single-point practical examinations, the authors argue, capture performance under artificial conditions but say little about whether a trainee can function independently in a real workplace over time. The EFPE inverts this logic. Rather than compressing assessment into one event, it spreads it across an extended period, embedding practical tasks within the trainee’s own institution and tying them directly to the needs of that institution. The result is an examination that doubles as a period of supervised, authentic professional work.

To understand how this model performs in practice, the research team conducted a reflective qualitative study spanning multiple medical schools. They gathered data from three distinct samples: a convenience sample of postgraduate students, and purposive convenience samples of faculty members and medical school stakeholders. Each group was interviewed using semi-structured interviews, a method that combines a consistent core of questions with the flexibility to probe unexpected themes as they arise. All interviews were audio recorded, transcribed verbatim, and subjected to thematic analysis, the standard qualitative technique of coding transcripts and iteratively grouping codes into themes and subthemes that capture the shared structure of participants’ experiences.

The thematic analysis surfaced three dominant themes across the three samples. The first concerned the format of the EFPE and how it differs from conventional examinations. The second addressed the examination’s impact on learners and institutions. The third mapped the challenges and areas for improvement that participants encountered. This structure allowed the researchers to compare perspectives across students, faculty, and stakeholders, revealing where the three groups converged and where their priorities diverged. Such triangulation is a key strength of qualitative designs that draw on multiple stakeholder samples, because it reduces the risk that findings reflect only one vantage point within a complex educational system.

On the question of format, participants consistently highlighted the alignment between EFPE tasks and institutional needs. In the EFPE model, the practical work a trainee undertakes is not an abstract exercise but a functional contribution to the hospital or school hosting them. This grounding in real institutional priorities is what the authors describe as the examination’s experiential character: learning by doing, with assessment woven into the doing. Participants also emphasized the positive role of feedback and mentorship, which in the EFPE framework are not afterthoughts but structural components. Continuous guidance during the assessment period means that errors become teaching moments rather than terminal failures, a stark contrast to the one-shot examination in which a single mistake can define the outcome.

The impact theme was the most emphatic. Participants reported that the EFPE led to significant skill development and confidence building among postgraduate students. This finding aligns with a well-established principle in medical education: competence in knowledge, skills, and attitudes develops through repeated, supported practice in authentic settings, not through isolated test events. Because the EFPE is process-focused, it rewards the trajectory of improvement rather than a single terminal performance. Students in the study were described as highly valuing this practical, structured approach, which they experienced as more meaningful and more transferable to their daily clinical responsibilities than traditional practical testing.

The study’s honesty about limitations is notable. Participants identified real costs to the model. Stress and workload emerged as significant concerns, since an extended examination necessarily demands sustained effort over weeks or months rather than a concentrated burst of preparation. Resource demands were another pressure point: longitudinal, mentorship-rich assessment requires faculty time, institutional infrastructure, and coordination that conventional examinations do not. Participants also flagged problems with topic selection and clarity, suggesting that in some cases the tasks chosen for the extended examination were not optimally defined or communicated. These findings matter because they temper enthusiasm with realism; an assessment innovation that ignores its own operational burden is unlikely to spread beyond enthusiastic early adopters.

From these challenges, the participants generated concrete recommendations. They suggested extending the duration of the EFPE, on the logic that a longer runway would distribute workload more evenly and allow deeper engagement with each task. They called for involving students in topic selection, which would increase the relevance of assessed work and give trainees a stake in shaping their own evaluation. They urged that continuous feedback and mentorship be maintained as non-negotiable features rather than optional extras. Finally, they stressed that tasks must remain relevant and sustainable, balancing educational value against the practical capacity of institutions to support them. Together, these suggestions sketch a roadmap for refining the model without abandoning its core principles.

The broader significance of the study lies in its contribution to the debate over longitudinal assessment in medical training. Assessment scholars have long argued that competence is too complex and too context-dependent to be validly measured at a single point in time, and programs worldwide have experimented with workplace-based assessment, portfolios, and longitudinal integrated clerkships. The EFPE adds a postgraduate-specific model to this landscape, one that explicitly fuses assessment with institutional service. By anchoring the examination in the host institution’s real needs, it addresses a persistent weakness of simulated assessments: the gap between what candidates do in an examination hall and what they must do on the ward. The study’s multi-school design suggests the model is not an idiosyncratic local experiment but a transferable framework, though the qualitative methodology means it captures perceptions of value rather than objective outcome measures such as subsequent clinical performance.

What emerges from the research is a portrait of assessment reform that is both ambitious and self-aware. The EFPE, as evaluated by Talaat, Wasfy, and Abouzeid, is highly valued for its practical, structured, and process-focused approach, and it demonstrably builds skill and confidence in the postgraduate students who experience it. At the same time, the study does not pretend the model is cost-free. Stress, workload, resources, and clarity of task design remain open problems, and the participants’ own proposals for longer durations, shared topic selection, and sustained mentorship indicate that the examination is still evolving. For medical educators watching the global shift from assessment of learning toward assessment for learning, the study offers a detailed case study of what that shift looks like when it is implemented across real institutions with real constraints, and a reminder that the future of medical examination may look less like a single afternoon in an examination hall and more like the practice of medicine itself.

Subject of Research: Longitudinal practical assessment in postgraduate medical education

Article Title: The extended functional practical examination (EFPE): an innovative model for advancing assessment for learning in postgraduate medical education

Article References: Talaat, W., Wasfy, N. F., & Abouzeid, E. (2026). The extended functional practical examination (EFPE): an innovative model for advancing assessment for learning in postgraduate medical education. BMC Medical Education. https://doi.org/10.1186/s12909-026-10345-0

Image Credits: AI Generated

DOI: 10.1186/s12909-026-10345-0

Keywords: medical education, postgraduate assessment, extended functional practical examination, longitudinal assessment, experiential learning, assessment for learning, thematic analysis, feedback and mentorship, clinical competence, Suez Canal University, BMC Medical Education, extended