AI Models Rate Dental Exam Questions Harder Than Students Do, NASA-TLX Study Finds

When artificial intelligence models sit the same dental examination as human students, they do not merely answer differently — they report feeling the questions differently. A new cross-sectional study from Qassim University in Saudi Arabia has, for the first time, asked ten multimodal AI models to rate their own mental workload on real objective structured practical dental examinations (OSPDEs) and compared those self-ratings with those of twenty fourth-year dental students. The results, published in Discover Education, reveal a striking asymmetry: AI models can detect that a task is difficult, but they appear largely blind to its affective load, reporting attenuated frustration even on the question formats where they performed worst.

The study, led by Sanaa N. Al-Haj Ali of the Department of Orthodontics and Pediatric Dentistry and Ra’fat I. Farah of the Department of Prosthetic Dental Sciences, moves beyond the familiar accuracy benchmarks that dominate AI evaluation in medical education. Accuracy alone, the authors argue, cannot characterize how AI models engage with assessment tasks. Workload — the cost of achieving a particular level of performance, emerging from the interaction between task requirements and the operator’s skills and perceptions — offers a complementary lens. In humans, working memory is limited in capacity and duration, and when task demands exceed this capacity, performance is compromised. An operator may maintain accurate performance at an unacceptably high mental cost, which is why performance and subjective workload must be measured independently. This dissociation makes workload a valuable evaluative dimension for AI systems, where models and students may achieve similar accuracy while experiencing vastly different processing demands.

To capture that dimension, the researchers turned to the National Aeronautics and Space Administration Task Load Index (NASA-TLX), the most widely used multidimensional subjective workload scale in human factors research. The original instrument comprises six subscales — Mental Demand, Physical Demand, Temporal Demand, Effort, Performance, and Frustration — each rated on a 21-point scale. For this study, the team selected three subscales tailored to the OSPDE context: mental demand as the primary indicator of cognitive processing, temporal demand given the timed, bell-ringer station format of the examinations, and frustration to probe whether AI models show any sensitivity to task-related affective load. Physical demand was excluded as inapplicable to both the examinations, which involve no psychomotor component, and to AI; performance was excluded to avoid overlap with measured accuracy; and effort was excluded because, in a task with no physical component, it reduces largely to mental effort. Subscales were analyzed individually rather than as a composite, consistent with the diagnostic approach recommended for identifying discrete workload sources.

The experimental setup was meticulous. Data were sourced from OSPDEs administered to twenty fourth-year dental students during a preclinical orthodontics and pediatric dentistry course in 2025. The question pool comprised seventy items across four formats: ten plain multiple-choice questions without visual support, and twenty short-answer questions, twenty multiple-choice questions, and twenty True/False questions, all supported by visuals such as clinical photographs, radiographs, illustrations, and tables. By clinical discipline, the pool included thirty-three pediatric dentistry items (47 percent) and thirty-seven orthodontics items (53 percent). Correct answers were validated against three standard reference texts, and all visuals passed a three-component quality checklist scored by a single faculty member whose intra-rater reliability exceeded Cohen’s kappa of 0.8.

Ten multimodal AI models were evaluated from three providers: four from Anthropic (Claude Sonnet 3.5, Claude Sonnet 4.5, Claude Haiku 4.5, and Claude Opus 4.6), three from OpenAI (GPT-5.2 Pro, GPT-5.1, and GPT-4o), and three from Google Gemini (Gemini 3 Pro, Gemini 2.5 Pro, and Gemini 2 Flash). All models were accessed via a paid subscription to POE between January 25 and March 5, 2026. A single investigator, blinded to study objectives and student performance data, inputted each question into the models via a new chat session to prevent context contamination, transcribing questions verbatim with visuals uploaded as JPEGs at 300 dpi using a zero-shot prompting approach. The team employed a two-step elicitation protocol: after providing its answer, the complete question–response pair was resubmitted to the model with a prompt requesting ratings on the modified NASA-TLX, each subscale rated on a seven-point scale from one (very low) to seven (very high). To assess intra-model response stability, each question was tested in three independent sessions, with final responses determined by majority rule and NASA-TLX ratings by median value across trials. Students completed the same modified NASA-TLX in print immediately following the examination, with a 100 percent response rate.

The accuracy results set the stage for the workload findings. Overall accuracy differed significantly across responders (chi-square = 95.309, p < .001): students achieved the highest score at 77.9 percent, followed by GPT-5.2 Pro at 74.3 percent and Gemini 3 Pro at 72.9 percent, while Claude Haiku 4.5 scored lowest at 44.3 percent. Only GPT-5.2 Pro, Gemini 3 Pro, Gemini 2.5 Pro, and Claude Opus 4.6 did not differ significantly from students. Performance gaps were largest on visual short-answer questions (chi-square = 44.676, p < .001), where several Claude models scored at or below 25 percent, and smallest on plain multiple-choice questions, where the difference was not statistically significant. Intra-model reliability, measured by intraclass correlation coefficients, ranged from 0.677 to 0.955 across all models and dimensions (all p < .001), with mental demand showing the highest consistency and frustration the most variability — a gradient the authors attribute to frustration capturing an emotional reaction that is inherently more state-dependent than ratings referencing concrete task properties.

The workload comparisons produced the study’s central contribution. Seven of ten models reported significantly higher mental demand than students, six reported significantly higher temporal demand, and nine reported significantly higher frustration. The greatest human–AI divergence occurred on visual short-answer questions, where nine of ten models reported significantly higher mental demand and all ten reported significantly higher temporal demand. Yet frustration told a different story on this format: only three models (Gemini 2.5 Pro, Gemini 3 Pro, and GPT-5.1) reported significantly higher frustration than students, compared with seven to nine models on other formats, despite visual short-answer questions producing the lowest AI accuracy. On plain multiple-choice questions, models performed comparably to students on both accuracy and mental demand, indicating that human–AI convergence is greatest when visual processing is absent and recognition-based options constrain response demands.

This dissociation between demand ratings and frustration on the hardest format is not paradoxical, the authors argue, once the cue sources underlying each dimension are distinguished. Mental and temporal demand are stimulus-facing judgments: their cues derive from observable task properties such as inferential steps, visual integration requirements, and time constraints. Frustration, by contrast, is a self-evaluative judgment that requires comparing output quality against some performance standard. In multiple-choice and True/False formats, fixed alternatives provide an implicit benchmark against which the model can gauge correctness; short-answer questions strip away this external scaffold entirely, leaving the model with impoverished evaluative cues. Koriat’s cue-utilization framework supports this interpretation: metacognitive judgments lose diagnostic sensitivity when available cues are restricted. Additionally, the higher processing demands on visual short-answer items may consume resources that would otherwise support self-evaluative reflection, analogous to Sweller’s finding that heavy cognitive load diminishes capacity for secondary processes. Hence, the models had ample cues to detect that the task was difficult, but insufficient cues to evaluate how poorly they performed.

The authors also caution against anthropomorphic readings of the frustration data. The NASA-TLX frustration item asks how insecure, discouraged, irritated, or stressed the respondent felt — wording that presupposes an affective state current models do not possess. A model’s response is therefore generated text about frustration rather than a report of an experienced emotion, and the researchers treat the affective subscale strictly as a behavioral output rather than as evidence of internal experience. A parallel caveat applies to temporal demand: language models generate output without an internal clock or representation of elapsed time, so the instruction that two minutes were available was processed as semantic content rather than experienced as time pressure. Elevated temporal demand ratings are therefore more plausibly an inference from task characteristics — item length, information density, reasoning steps, and visual parsing load — than a report of temporal stress. This mirrors the broader pattern: models tracked content-derived indices of difficulty while showing attenuated responses on the affective dimension, which carries no comparable task signal.

Correlational analyses between workload and accuracy revealed a predominantly inverse relationship among both students and AI models, though among students significant correlations were weak and confined to frustration (rs = −0.08, p < .01). Among AI models, the pattern was architecture-dependent: Claude Sonnet 4.5 and Claude Opus 4.6 showed significant negative correlations across all three dimensions overall, while GPT-4o, Claude Sonnet 3.5, and the Gemini models showed none. Claude Haiku 4.5 showed significance for temporal demand (rs = −0.35) and frustration (rs = −0.40), and GPT-5.2 Pro for temporal demand (rs = −0.30) and frustration (rs = −0.33). At the format level, significant negative correlations were most concentrated on visual multiple-choice and visual short-answer items and least on visual True/False items and plain multiple-choice questions, suggesting that the diagnostic sensitivity of AI workload ratings scales with format complexity. This variability has practical significance: models with significant negative load–accuracy associations and acceptable accuracy — notably Claude Opus 4.6 and GPT-5.2 Pro — could serve as automated item difficulty estimators, flagging disproportionately demanding questions before student administration during examination development.

The findings carry implications for educators designing assessments intended to differentiate human from AI performance. Because human–AI convergence was greatest on plain multiple-choice questions and diverged most on visually integrated open-ended formats, educators seeking to distinguish human from AI capability should favor visually integrated open-ended question types. The study also establishes workload profiling as a complementary dimension of AI evaluation that extends beyond accuracy, offering insight not only into whether AI can answer correctly but into how it represents the difficulty of doing so — and where that representation breaks down.

Several limitations warrant acknowledgment. AI answers were determined by majority rule across three sessions whereas students had a single attempt, an asymmetry that, for accuracy, can only maintain or improve an AI’s score relative to a single attempt — meaning the finding that students outperformed most models is conservative. The student sample comprised twenty participants from a single institution, representing the entire eligible cohort but limiting generalizability, and the seventy-item question pool was smaller than licensing examinations. Potential data contamination cannot be fully excluded, as the training corpora of commercial models are proprietary, though the authors note that the study’s central finding — a systematic cross-item pattern in which workload tracked question format and visual complexity with a consistent demand–affect dissociation — would not be expected from memorization of individual images. Whether AI workload ratings reflect genuine metacognitive self-monitoring or statistically patterned output remains open; the reliability and correlational data favor a structured interpretation but cannot establish introspective capacity. Future work, the authors suggest, should compare both parties under identical single-attempt conditions and explore whether format-specific time allocations reduce the human–AI gap on open-ended items.

Subject of Research: Science News

Article Title: NASA TLX workload profiles of multimodal artificial intelligence models and dental students during objective structured practical dental examinations

Article References: Al-Haj Ali, S. N., & Farah, R. I. (2026). NASA TLX workload profiles of multimodal artificial intelligence models and dental students during objective structured practical dental examinations. Discover Education, 5(1), Article 835. https://doi.org/10.1007/s44217-026-02116-2

Image Credits: AI Generated

DOI: 10.1007/s44217-026-02116-2

Keywords: artificial intelligence, NASA-TLX, workload, dental education, multimodal AI, large language models, metacognition, cognitive load, objective structured practical dental examinations, AI evaluation, NASA, profiles