Mixed-methods study finds no significant score difference between two multiple-choice formats among first-year optometry students

A mixed-methods evaluation involving 34 first-year optometry students found no statistically significant difference in average scores between confidence-based answer tests and conventional single best answer multiple-choice tests. According to the published abstract, the median difference between the formats was 1.25% (p = 0.31, Kruskal–Wallis test).

The students completed a 50-question multiple-choice test, with each question answered using both approaches, allowing a within-participant comparison. The researchers analysed differences between the two paradigms using the Bland–Altman method of agreement and linear regression, and examined whether either format showed systematic bias. After the test, students completed a survey combining Likert-scale items with free-response questions about their experiences and perspectives. The qualitative responses were analysed thematically.

The survey results indicated that students had no strong preference for either approach. Thematic analysis identified themes of confidence, risk-taking behaviours, test-taking motivation and the learning journey. The authors concluded that confidence-based answer tests offered opportunities for reflective practice and described them as a viable equivalent and alternative to conventional single best answer questions. In the abstract, confidence-based tests are described as reducing the need to guess when participants cannot distinguish between two or more alternatives they favour equally, rather than as a format in which respondents select multiple favoured alternatives. The authors also proposed that recording additional information about confidence could provide insight not otherwise visible in a conventional test, but the reported evaluation did not directly establish that confidence-based questions produced more reliable measures of knowledge or improved learning.

The findings are bounded by the study’s small, cohort-based setting: 34 first-year optometry students taking one 50-question test. The reported score analysis did not show a significant advantage for either paradigm, and the qualitative finding about reflection did not demonstrate that it caused better performance or learning. Detailed sampling procedures, statistical results beyond those reported and the fuller qualitative evidence cannot be assessed from the abstract alone. This report is based on the supplied article abstract and early published version, which states that the article remains subject to further edits.

Subject of Research: Comparison of confidence-based and single best answer multiple-choice formats in first-year optometry students

Article Title: A mixed methods evaluation of the effect of confidence-based versus single best answer multiple-choice questions on student performance and the learning journey

Article References: Chong, L. X., Hockley, N., Wood-Bradley, R. J., & Armitage, J. A. (2026). A mixed methods evaluation of the effect of confidence-based versus single best answer multiple-choice questions on student performance and the learning journey. BMC Medical Education. https://doi.org/10.1186/s12909-026-10458-6

Image Credits: AI Generated

DOI: 10.1186/s12909-026-10458-6

Keywords: mixed-methods evaluation, medical education, optometry students, multiple-choice assessment, confidence-based tests, single best answer tests, test scores, student perceptions, reflective practice, Bland–Altman analysis, mixed, methods