Natural Gestures Beat Menus in AR Training, but Speed Stays the Same

When soldiers, medics, or industrial technicians train in augmented reality, the way they interact with virtual content can determine whether the experience feels seamless or frustrating. A new controlled experiment from the University of Central Florida, published in Frontiers in Virtual Reality, provides some of the most direct evidence yet on this question. Researchers Claire L. Hughes and Waldemar Karwowski compared two fundamentally different interaction paradigms—embodied, multimodal natural interaction and the traditional WIMP interface of windows, icons, menus, and pointing devices—within a high-stakes military medical training scenario, and found that natural interaction delivered clear advantages in usability, presence, and execution accuracy, even though it did not make people faster or reduce their mental workload.

The study matters because augmented reality is expanding rapidly across industry, retail, manufacturing, and education, yet systematic reviews have repeatedly noted that user experience design is often overshadowed by hardware capabilities and visualization techniques. Interaction design sits at the center of this problem. In AR, digital content is merged with the physical environment in real time and in spatially accurate ways, so the interface must bridge two worlds. The dominant assumption in the field has been that natural interaction techniques—gestures, gaze, voice, and direct manipulation of virtual objects—suit AR better than the symbolically mediated menus inherited from two-dimensional desktop computing. But direct, controlled comparisons between the two paradigms have been surprisingly rare, with most evidence coming from evaluations of single, isolated systems.

To close that gap, the researchers designed a within-subjects experiment in which every participant experienced both interaction conditions. Thirty adults, recruited from university and community populations and ranging widely in age and prior AR experience, completed a Tactical Combat Casualty Care training simulation delivered on a Meta Quest 3 head-mounted display. The scenario, built on Design Interactive’s AUGMED platform, focused on identifying and treating massive hemorrhage in a battlefield setting—a task chosen precisely because it demands correct execution of a sequence of physical actions under time pressure, the kind of high-stakes procedural training where interaction quality matters most. The tasks were identical across conditions in objectives, sequence, and content, so any differences in outcomes could be attributed to the interaction modality itself.

The two conditions embodied opposite design philosophies. In the natural interaction condition, participants used gesture-based manipulation, gaze-based selection, and voice commands—inputs chosen because each preserves a direct, continuous correspondence between a physical action and its digital effect, such as reaching toward an object to select it. In the WIMP condition, the same tasks were presented through a graphical interface of floating windows, icons, and menus rendered spatially but retaining a two-dimensional, symbolic interaction structure that required indirect manipulation of virtual content. Condition order was counterbalanced across participants, and a priori power analysis confirmed that thirty participants were sufficient to detect moderate effects with paired-samples t-tests at conventional statistical thresholds.

The measurement strategy combined validated subjective instruments with objective behavioral metrics. Participants completed the System Usability Scale after each condition, the NASA Task Load Index to capture mental, physical, temporal, and other dimensions of workload, and the Presence Questionnaire to assess how strongly they felt present in the augmented environment. The Immersive Tendencies Questionnaire was administered beforehand as a baseline measure of each person’s general propensity for immersion. On the objective side, the experimenters recorded task completion time, misclicks and navigation errors, missed procedural steps, corrective actions, requests for clarification, and observed ratings of interaction smoothness and efficiency during task execution.

The results told a nuanced story. Natural interaction produced significantly higher usability scores, averaging 79.83 compared with 72.67 for the WIMP interface, a statistically significant difference with a moderate effect size. A nonparametric sensitivity analysis confirmed this finding was robust. Presence showed an even stronger advantage, with participants reporting a significantly greater sense of being there under natural interaction, an effect in the medium-to-large range. Crucially, the presence advantage held regardless of which condition participants tried first, and it was not explained by individual differences in immersive tendency, which showed no significant correlation with presence in either condition. The interaction technique itself, rather than a participant’s personality, appears to have driven the immersion benefit.

Objective performance measures reinforced the pattern. Participants committed significantly fewer navigation errors, missed fewer procedural steps, required fewer corrective actions, and demonstrated smoother observed interactions under the natural paradigm, with effect sizes ranging from small to medium. Yet several measures showed no significant difference at all: task completion time, requests for clarification, and overall observed efficiency were statistically equivalent between conditions. Cognitive workload, measured by the NASA-TLX, also showed no significant difference between paradigms, despite mean scores trending slightly lower for natural interaction. In short, natural interaction made people more accurate and made the experience feel better, but it did not make them quicker or lighter on mental demand during initial use.

One of the most instructive findings emerged from the exploratory order analysis. Perceived workload was significantly lower during participants’ second experimental session regardless of which interaction condition they were using, with a mean reduction of 5.70 points on the NASA-TLX. Similar order effects appeared for task completion time, clarification requests, and subjective efficiency ratings. The researchers interpret this as a learning or familiarization effect: participants devoted substantial mental effort to understanding the task, the AR environment, and the interaction paradigm during their first exposure, and that burden diminished with experience. Because no significant interaction between condition and order was observed, the effect reflects acclimation to the setup rather than an inherent advantage of either interface—a reminder that in immersive technology research, the novelty of the medium itself can dominate early workload measurements.

Taken together, the findings suggest that the principal value of natural interaction in AR lies in improving the overall quality of the user experience rather than maximizing efficiency. The authors argue this distinction is especially important in domains such as training, maintenance, healthcare, and decision support, where accuracy, user confidence, and sustained engagement matter alongside productivity. Natural interaction appears to encourage deliberate, embodied movements that prioritize accuracy and confidence over raw speed, and because both interfaces in the study were designed to support successful task completion, opportunities to observe large efficiency differences may have been limited. The researchers also acknowledge limitations: the results come from a single training application on one hardware platform, the sample was drawn from university and community populations rather than operational end users, and the study relied on domain-general instruments rather than emerging AR-specific evaluation frameworks that might better capture spatial and embodied dimensions of interaction.

Future work, the authors suggest, should replicate these findings across diverse AR applications and populations, investigate why natural interaction improves usability and presence without reducing perceived workload, and examine how the benefits evolve over long-term use, in collaborative settings, and in more complex operational scenarios. Disentangling the individual contributions of gesture, gaze, and voice would also help designers of next-generation AR interfaces decide where to invest. For now, the message for the field is clear: if the goal is interfaces that are intuitive, immersive, and forgiving of human error, designing interaction techniques that align with people’s natural behaviors offers measurable advantages—provided that speed alone is not the yardstick.

Subject of Research: A controlled comparison of natural versus WIMP-based interaction paradigms in augmented reality combat casualty care training

Article Title: Evaluating interaction methods in augmented reality training

Article References: Hughes, C. L., & Karwowski, W. (2026). Evaluating interaction methods in augmented reality training. Frontiers in Virtual Reality, 7, Article 1939058. https://doi.org/10.3389/frvir.2026.1939058

Image Credits: AI Generated

DOI: 10.3389/frvir.2026.1939058

Keywords: augmented reality, natural interaction, WIMP interface, user experience, presence, cognitive workload, usability, Tactical Combat Casualty Care, Meta Quest 3, training simulation, interaction design, human-computer interaction