Video review has become a widely used method for teaching communication skills across undergraduate and postgraduate medical education. By watching recordings of their own consultations, trainees can observe behaviors that are nearly invisible in the moment, such as interruptions, missed cues from patients, or awkward transitions between phases of the clinical encounter. Yet the technique depends on having a shared vocabulary between the observer and the trainee. Without a structured framework, feedback can drift into vague impressions, and residents may struggle to translate general praise or criticism into concrete changes in behavior. In Japan, the researchers note, few assessment tools have been developed specifically to support video-based reflection in general practice residency training, particularly through clearly described development processes that other educators could scrutinize or replicate.

The team chose an unusual but pragmatic development strategy: rapid prototyping combined with the nominal group technique, a structured consensus method in which experts generate, discuss, and rank ideas in successive rounds. Rather than attempting to design a definitive instrument in a single pass, the researchers treated the tool itself as a prototype to be tested, revised, and refined against real clinical material. This approach mirrors practices familiar from software engineering, where iterative cycles of building, testing, and feedback replace the hope of getting everything right on the first attempt. In medical education, where assessment instruments must balance theoretical grounding with practical usability in busy training environments, such iterative refinement can reveal problems that purely theoretical design would miss.

The first stage involved a systematic sweep of the existing literature. The researchers reviewed established communication assessment instruments and educational frameworks, using the Kalamazoo Consensus Statement, an influential international consensus document on doctor-patient communication, as a key reference point. From twelve existing instruments and frameworks, including well-known tools such as the MAAS-Global rating list for consultation skills, the Calgary-Cambridge Observation Guide, the Four Habits Coding Scheme, the Arizona Clinical Interview Rating Scale, and the SEGUE framework, they extracted a total of 127 individual evaluation items. This extraction step ensured that the new tool would stand on the accumulated evidence of decades of communication skills research rather than on the preferences of a single group of developers.

Those 127 items then entered the consensus machinery of the nominal group technique. Two sessions with experts in general practice were convened to generate, refine, and consolidate the raw material. Through this process and subsequent expert review, the 127 items were distilled into 41 behavioral descriptors, each phrased to describe an observable action that could be identified on a video recording. The reduction from 127 to 41 reflects a central tension in assessment design: an instrument with too many items becomes unwieldy for routine use, while one with too few sacrifices the granularity needed for meaningful feedback. Behavioral descriptors, which anchor each item to something a reviewer can actually see or hear, are considered essential for formative assessment, where the goal is not to grade the trainee but to guide improvement.

To test whether the surviving items mattered to practicing physicians, the team conducted a web-based survey involving 93 physicians. Respondents rated the importance of each of the 41 items and indicated the stage of training at which the corresponding competency would be expected to be attained. The survey confirmed the importance of all 41 items, providing quantitative support that the consensus process had not filtered out anything practitioners considered peripheral. The question about training stage added a developmental dimension, allowing the tool to distinguish between skills expected early in residency and those that mature later, an important consideration for formative assessment that aims to meet trainees where they are.

With the item set validated, the researchers reorganized the 41 descriptors into 15 domains aligned with the structure of the medical interview itself. This organization means that a reviewer watching a recorded consultation can move through the tool in the same sequence as the encounter unfolds, from opening the consultation and gathering information through to explanation, planning, and closing. The final product is a rubric-based assessment tool with three developmental levels, allowing supervisors to characterize a resident’s performance on each item as progressing along a defined trajectory rather than simply present or absent. Rubrics of this kind serve a dual purpose: they standardize observation across different reviewers, and they give trainees an explicit picture of what growth looks like.

Critically, the prototyping did not stop at the drafting table. The team conducted iterative refinement using video recordings of actual outpatient consultations by residents, with appropriate consent obtained from both residents and patients and all data handled with strict attention to confidentiality and anonymization. Testing the draft items against genuine clinical encounters, rather than simulated scenarios, exposed the practical realities of video review: consultations in general practice are messy, brief, and shaped by the unpredictability of real patients. The study received ethics approval from the Mie University Ethics Review Committee and was conducted in accordance with the principles of the Declaration of Helsinki, with written informed consent from participants in the group sessions and an approved consent procedure for the survey.

The authors are careful about what the tool has and has not yet demonstrated. GP-VRAT provides a shared framework for observation and feedback and may support reflective learning through video review, but its feasibility, reliability, validity, and educational usefulness in practice remain to be examined in further studies. Reliability, in the psychometric sense, refers to whether different observers using the tool reach similar judgments on the same recording; validity concerns whether the tool actually measures the communication constructs it claims to measure; and educational usefulness encompasses whether its use genuinely changes resident behavior over time. These are the standard hurdles any new assessment instrument must clear, and the developers present GP-VRAT as a starting point rather than a finished verdict.

The significance of the work extends beyond Japan. General practice and primary care training programs worldwide grapple with the same challenge of assessing communication in authentic settings, and the transparent description of the development process, from literature extraction through consensus sessions, survey validation, and video-based prototyping, offers a template other educators can adapt. As video recording becomes increasingly routine in clinical training environments, tools like GP-VRAT may determine whether those recordings become genuine engines of reflective learning or merely an archive of awkward moments. For now, the researchers have provided the framework; the next chapters of evidence will come from the training rooms where residents and supervisors put it to work.

Subject of Research: Development of a video review assessment tool for communication skills training in general practice residency

Article Title: Development of the General Practice Video Review Assessment Tool (GP-VRAT): a rapid prototyping approach to formative assessment of communication skills in residency training

Article References: Development of the General Practice Video Review Assessment Tool (GP-VRAT): a rapid prototyping approach to formative assessment of communication skills in residency training. (n.d.). https://doi.org/10.1186/s12909-026-10435-z

Image Credits: AI Generated

DOI: 10.1186/s12909-026-10435-z

Keywords: GP-VRAT, video review, communication skills, general practice residency, formative assessment, medical education, nominal group technique, rapid prototyping, reflective learning, Kalamazoo Consensus Statement, rubric-based assessment, primary care