On-Demand Video Lectures Linked to Higher Medical Test Scores in Japanese Cohort Study

Medical students who learned through on-demand video lectures scored significantly higher on computer-based knowledge tests than students taught before their introduction, according to a retrospective cohort study published in BMC Medical Education. However, the same study found no significant difference on a hands-on clinical skills examination, and its authors caution that the observed associations cannot be attributed solely to the shift to video learning.

The research, led by Manami Yamaguchi and colleagues at a Japanese national university, took advantage of the widespread adoption of on-demand lectures since the COVID-19 pandemic. The authors note that the association between on-demand lectures and learning had not previously been investigated in an objective and comprehensive way. To address this, the team compared two cohorts of fourth-year medical students: one that completed pre-clinical education before video lectures were introduced, and one that studied after their implementation. In total, 643 test records were available, and after applying exclusion criteria, 629 records were analyzed.

Learning outcomes were measured using the Pre-Clinical Clerkship Common Achievement Tests, which include two components. The first is Computer Based Testing, or CBT, a knowledge examination. The second is the Objective Structured Clinical Examination, or OSCE. The researchers analyzed the data with a generalized linear model that included group membership, grade point average, and the interaction between the two, allowing them to test whether the relationship between academic performance and test results changed between cohorts.

The results showed a pattern favoring the video lectures group, but with important nuances. Students in the post-implementation cohort had a significantly higher GPA than their pre-implementation counterparts. Their CBT scores were also significantly higher, although the effect size was small, with a Cohen’s d of 0.411. By contrast, OSCE scores showed no significant difference between the two groups.

The analysis also revealed a significant interaction between GPA and group: the association between GPA and CBT performance was stronger in the cohort that studied after on-demand video lectures were introduced. The authors interpret this as evidence that the relationship between academic performance and knowledge assessment outcomes may differ between cohorts, rather than simply that one group learned more than the other.

The study’s conclusions are deliberately restrained, and the authors are explicit about why. Because the design was retrospective, students were not randomly assigned to learning formats, and GPA itself differed between the groups, the observed associations cannot be attributed solely to on-demand video lectures. The team calls for further prospective studies with adequate control of confounding factors before firmer conclusions can be drawn.

The divergence between CBT and OSCE results is a notable feature of the findings. The apparent difference between cohorts appeared on a knowledge-based test but not on the clinical skills examination, indicating that the two assessment formats behaved differently across the two eras of education. The authors do not offer an explanation for this divergence in the reported results, and the study was not designed to establish why knowledge scores and clinical skills scores moved differently.

For now, the study stands as a sizable attempt to quantify what happened when a medical school moved lectures online. Its use of standardized achievement testing, a sample of 629 records, and formal statistical modeling of the GPA-by-group interaction gives the findings more texture than simple before-and-after comparisons. But the authors’ own caution is the takeaway that matters most: higher test scores in the video lecture era are an association, not proof, and only prospective research can determine whether on-demand lectures genuinely improve learning or simply coincide with a changed relationship between grades and test performance.

Subject of Research: Association between on-demand video lectures and Common Achievement Tests scores in pre-clinical medical education

Article Title: Association between on-demand video lectures and Common Achievement Tests scores in pre-clinical education

Article References: Yamaguchi, M., Sakamoto, M., Mitsutake, T., Fukumori, N., Yamashita, S., Ueda, M., Kimoto, A., Yamazaki, K., Koba, Y., & Oda, Y. (2026). Association between on-demand video lectures and Common Achievement Tests scores in pre-clinical education. BMC Medical Education. https://doi.org/10.1186/s12909-026-10296-6

Image Credits: AI Generated

DOI: 10.1186/s12909-026-10296-6

Keywords: medical education, on-demand video lectures, Common Achievement Tests, CBT, OSCE, retrospective cohort study, COVID-19, GPA, pre-clinical education, BMC Medical Education, Association, between