The research team, led by Deena M. Hamza of the University of Alberta alongside colleagues from Queen’s University, McMaster University, the University of Ottawa, the University of British Columbia, and the Royal College itself, conducted a rapid evaluation study across four Canadian internal medicine residency programs. Rather than waiting years for outcome data, the team used a utilization-focused evaluation approach, gathering data through interviews and focus groups with residents, faculty, and program administrators at each site. Their qualitative thematic analysis was guided by the Core Components Framework for evaluating competency-based medical education implementation, a tool designed to compare what programs intended to do against what they actually did, and to identify the adaptations programs made along the way.

The framework allowed the researchers to systematically examine five key elements of Competence by Design: outcome competencies, sequenced progression, tailored learning experiences, competency-focused instruction, and programmatic assessment. By comparing stated intentions with reported experiences, the team could identify where implementation was faithful to the design and where programs had drifted or adapted. This distinction matters enormously for a national reform initiative. If programs are quietly reshaping the model to fit local constraints, that adaptation may be reasonable, or it may signal a gap between theory and practice that needs addressing before the reform’s benefits can be realized.

On the positive side, the findings were striking. Programs reported more frequent feedback and more direct observation of trainees than under the previous system. This is a fundamental shift. Under time-based training, feedback often arrived sporadically, sometimes only at the end of a rotation, when the opportunity to act on it had already passed. Competence by Design was designed to change that by embedding assessment into daily clinical work, and the study suggests it is working. Residents and faculty described enhanced coaching relationships, with faculty taking on roles closer to mentors and guides than examiners. Programs also engaged in proactive curricular review, actively realigning their training activities with the outcome competencies that define success in the new model.

These successes are not trivial. Direct observation of trainees is one of the most powerful tools in medical education, allowing faculty to see what residents can actually do rather than relying on secondhand reports or end-of-rotation impressions. Frequent feedback, delivered close to the moment of performance, gives residents the chance to adjust in real time. Coaching relationships, in which faculty invest in a resident’s development over time rather than simply judging them, have been shown to accelerate learning and improve trainee satisfaction. The fact that these elements emerged as key successes across four different programs suggests that the core design of Competence by Design is resonating with the people who use it daily.

But the study also surfaced prominent challenges that the reform’s architects will need to confront. The most significant was confusion about expectations. Residents and faculty reported unclear expectations regarding what performance was required at different training stages. This reflects a gap in shared understanding of sequenced progression, one of the five core components of the model. In theory, sequenced progression means residents advance through defined stages, each with its own expectations, and faculty know what to look for at each stage. In practice, the study found that this shared understanding had not fully taken hold. Faculty were uncertain what to expect from a resident in one stage versus another, and residents were uncertain what was expected of them. This ambiguity undermines the precision that competency-based education is supposed to deliver.

The second major challenge was technological. The electronic assessment systems that programs relied on to track resident progress had significant limitations. These platforms were not designed to efficiently capture and display the granular data that competency-based education generates, making it difficult for faculty and program administrators to use assessment data for tracking progress. The result was a system that generated data but did not make it actionable. This is a familiar problem in health information technology: platforms built for one purpose often struggle to serve the needs of a new paradigm. The study suggests that without better technological support, the data-rich promise of competency-based education will remain partially unrealized.

The third challenge was the sheer volume of assessment. Competence by Design increases the frequency of assessment, which is central to its design, but this frequency comes at a cost. Faculty described increased assessment burden and assessor fatigue, a phenomenon that has been documented in other competency-based education implementations. When faculty are asked to complete more assessments more often, the risk is that assessments become perfunctory, completed to satisfy a requirement rather than to genuinely evaluate performance. This drift toward bureaucratic box-ticking has been noted in the literature as a misalignment between curriculum theory and clinical reality, and the study’s findings suggest it is a real risk in internal medicine programs.

The researchers conclude that early evaluation of Competence by Design in internal medicine has yielded notable improvements in feedback culture, direct observation, and curriculum alignment, but that significant cultural and logistical challenges persist. They call for ongoing program refinements, faculty development, and technological support to realize the full potential of the model. Faculty development is particularly important, because the success of competency-based education depends on faculty understanding the new expectations and being equipped to assess against them. Without sustained investment in developing faculty, the confusion about expectations documented in this study is likely to persist.

What makes this study valuable is its honesty. Rather than presenting Competence by Design as either a triumph or a failure, the team documented both the genuine gains and the persistent difficulties. The findings suggest that the reform is neither a revolution completed nor a promise broken, but a work in progress. The feedback culture is improving. Direct observation is increasing. Curricula are being realigned. But the shared understanding of what competency means at each stage, the technology to track it, and the assessment workload to sustain it remain unresolved. For a national reform initiative now several years into implementation, this kind of rapid, honest evaluation is exactly what is needed to guide the next phase of refinement. The study offers a roadmap for what programs, faculty, and the Royal College should prioritize as Competence by Design matures: clarify expectations, invest in faculty development, fix the technology, and manage the assessment burden. Those four priorities may determine whether the reform ultimately delivers on its promise to produce better-trained physicians.

Subject of Research: Exploring the Intentions, Experiences, and Adaptations of Postgraduate Internal Medicine Programs in the Implementation of Competence by Design

Article Title: Exploring the Intentions, Experiences, and Adaptations of Postgraduate Internal Medicine Programs in the Implementation of Competence by Design

Article References: Hamza, D. M., Ladak, Z., McGuire, N., Oswald, A., Katz, S., Martin, L., Vegas, D. B., Gilchrist, T., Yelle, D., van Melle, E., Frank, J. R., Cheung, W. J., Derby, B., Dalseg, T. R., Cooke, L., & Hall, A. K. (2026). Exploring the Intentions, Experiences, and Adaptations of Postgraduate Internal Medicine Programs in the Implementation of Competence by Design. Journal of General Internal Medicine. https://doi.org/10.1007/s11606-026-10795-8

Image Credits: AI Generated

DOI: 10.1007/s11606-026-10795-8

Keywords: medical education, competency-based education, residency training, internal medicine, postgraduate training, assessment, Canada, health professions education, Exploring, Intentions, Experiences, Adaptations