Now a team of researchers at the ZEISS Vision Science Lab at the University of Tübingen and Carl Zeiss Vision International GmbH has developed and rigorously tested a real-time Error Prevention System (EPS) that tackles the Midas touch problem head-on. Rather than relying on predefined error categories or explicit confirmation steps, the system treats selection errors as anomalies within the continuous temporal gaze signal. The approach, described in a study published in Frontiers in Virtual Reality, is grounded in a simple but powerful observation: correct selections are characterized by relatively stable and consistent gaze patterns, whereas erroneous selections often involve increased uncertainty, corrective movements, or unstable fixations. By learning what typical correct selections look like, the system can flag deviations from this learned representation as potential errors before they cause task-relevant consequences.

The technical heart of the EPS is a Temporal Convolutional Network Autoencoder (TCNAE), a model architecture that has already proven effective for anomaly detection in time series data. An autoencoder is a neural network trained to reconstruct its own input: it learns a compressed representation of typical patterns and then attempts to rebuild the original signal from that representation. Crucially, the TCNAE was trained exclusively on gaze dynamics associated with correct selections. Because anomalous selections were never part of the training data, the network should struggle to reconstruct them, resulting in a higher reconstruction error. This error serves as a measure of selection validity: when the reconstruction error exceeds a fixed threshold, the selection is classified as anomalous and the action is interrupted before completion. The threshold was set at the 95th percentile of reconstruction errors obtained from correct selections in the training set, a compromise between detecting anomalous selections and limiting false rejection of correct ones.

The input representation reflects careful engineering choices informed by the realities of real-time eye tracking. Rather than using angular distances between consecutive gaze vectors, which are sensitive to variations in temporal spacing caused by dropped frames or irregular sampling rates, the researchers used angular velocity, which normalizes the angular change by the time difference between samples. The network operates on the last 37 gaze vectors captured in a temporal window centered around the selection event. At a sampling rate of 90 Hz, this window spans approximately 200 milliseconds before and after the selection. The post-selection interval was deliberately kept short: human corrective responses typically occur with a delay of around 300 milliseconds, so using a shorter interval allows the system to respond earlier without relying on explicit user correction. Including gaze data before the selection provides additional context, ensuring the model captures the transition into the selection event.

Training the system required substantial data from a previous experiment conducted in the same virtual environment. The researchers captured all correct selection events from 52 participants, collecting 4,842 correct selections for Dwell Time, 4,725 for Head and Gaze, and 4,926 for Nod. Hyperparameters were determined through five-fold cross-validation performed at the participant level, meaning all samples from a given participant were assigned to the same fold to prevent overlap of participant-specific gaze behaviour between training and validation data. Cross-validation was performed separately for each interaction method, and results were aggregated across methods to find the hyperparameter configuration achieving the best overall performance. A separate network was trained for each interaction method to account for differences in gaze behaviour across techniques. The network was trained for 2,000 epochs with a batch size of 4,000 using the Adam optimizer and Mean Squared Error as the loss function.

To evaluate the EPS in practice, the researchers integrated it into a VR visual search game and recruited 37 participants (26 women and 11 men, mean age 26.14 years) from the University of Tübingen and the surrounding area. Most reported little or no prior experience with eye tracking or VR. The game required participants to identify a target among flying robots positioned in a square-shaped virtual room. At the beginning of each round, robots moved to one of approximately 18 predefined positions arranged within a 180-degree field in front of the user. Once a robot reached its position, a sphere appeared displaying a character: the target was indicated by the letter C, while distractors were indicated by O. Destroying the correct target earned between 5 and 20 points depending on response time, whereas destroying a distractor resulted in a 21-point penalty. Each round lasted 30 seconds, and the scoring scheme was designed to encourage both accuracy and speed under time pressure.

Three gaze-based selection methods were compared, each evaluated with and without the EPS. Dwell Time allowed participants to select a target by fixating on it for a specified duration, set to 0.4 seconds. Gaze and Head required both gaze direction and head orientation to remain aligned with the target for 0.3 seconds. Nod involved a gaze fixation followed by a nod gesture to confirm the selection. When the EPS was active, selections were validated using the TCNAE while the action was being executed and before task-relevant consequences such as target destruction and score updates occurred. Model inference required less than 1 millisecond, negligible compared to the approximately 200-millisecond post-trigger acquisition period needed to complete the input window. If a selection was classified as anomalous, the action was interrupted before completion. The experiment used an HTC Vive Pro Eye head-mounted display with a built-in Tobii eye tracker, and gaze data were accessed once per rendered frame at approximately 90 Hz.

The results are striking. With the EPS active, participants made markedly fewer incorrect selections across all three methods. For Dwell Time, the median number of incorrect selections dropped from 22 without EPS to 1 with EPS, a reduction of up to 95 percent. A similar effect was observed for Head and Gaze, with medians falling from 8 to 1. For Nod, the reduction was statistically significant but the median remained 0 in both conditions, indicating that the benefit was driven by the few participants who made errors rather than by a broad shift in behaviour. Participants also gained substantially more points when the EPS was active: the median score for Dwell Time increased from 435 to 925, and for Head and Gaze from 745 to 819. No significant difference was observed for Nod, where medians were nearly identical at 787 without EPS and 789 with EPS. The EPS itself achieved strong classification performance: for Dwell Time, 79.4 percent of correct selections were classified correctly and 92.8 percent of incorrect selections were classified as incorrect; for Gaze and Head, the corresponding figures were 73.0 and 82.1 percent; for Nod, 90.8 and 60.7 percent.

Subjective evaluations were equally revealing. Participants rated the system positively overall, reporting improved confidence and usability. Of the 37 participants, 31 said the EPS was helpful, only 1 rated it as not helpful, and 5 were not sure. Gaze and Head was the most frequently preferred interaction method, chosen by 16 participants, followed by Nod with 10 and Dwell Time with 11. Questionnaire analysis revealed significant differences between interaction methods for helpfulness, trust, confidence boost, and preference, with the Nod method receiving significantly lower ratings than Dwell Time and Head and Gaze across these dimensions. No significant differences were found for frustration or disruption, where median ratings were similar and generally low. Notably, participants’ ability to recognize whether the EPS was active was only slightly above chance level (50 percent) across methods, suggesting the system worked largely in the background without disrupting interaction. Interestingly, even participants with comparatively low accuracy scores stated they appreciated the system, suggesting that occasional false rejections were not perceived as disruptive, since rejected targets could be immediately reselected while visual attention remained focused on them.

The findings demonstrate that anomaly-based error prevention can effectively reduce incorrect gaze-based selections in real time without disrupting interaction, addressing a challenge that has persisted since the earliest days of eye movement-based interaction. However, the variability across interaction methods and users highlights that a uniform approach may not be sufficient. System performance varied across participants: while it was nearly perfect for some, it was less reliable for others, suggesting that individual gaze behaviour plays an important role in error detection. The authors note that previous research has shown gaze behaviour is highly individual and context-dependent, pointing toward the need for adaptive approaches, such as dynamic adjustment of thresholds or detection strategies to better accommodate individual characteristics and interaction styles. The study also has limitations: the findings relate specifically to a visual search task in VR and may not transfer directly to other applications, and the system was trained to distinguish selection correctness rather than directly infer user intention. Future work should validate the system in a broader range of environments, including AR and assistive technologies, and incorporate independent measures of intention. Nevertheless, the results represent a significant step toward gaze-based interfaces that are not only intuitive but also trustworthy, bringing the promise of truly hands-free interaction in VR, AR, and beyond closer to reality.

Subject of Research: Real-time anomaly detection for preventing unintended gaze-based selections in virtual reality

Article Title: A real-time error prevention system for gaze-based interaction in virtual reality based on anomaly detection

Article References: Severitt, B. R., Sauer, Y., Castner, N., & Wahl, S. (2026). A real-time error prevention system for gaze-based interaction in virtual reality based on anomaly detection. Frontiers in Virtual Reality, 7, Article 1918264. https://doi.org/10.3389/frvir.2026.1918264

Image Credits: AI Generated

DOI: 10.3389/frvir.2026.1918264

Keywords: gaze-based interaction, virtual reality, anomaly detection, Midas touch problem, error prevention, eye tracking, temporal convolutional network, autoencoder, dwell time, human-computer interaction, assistive technology, user experience

The study draws on the Multi-Modal Speed Dating corpus, a collection of recorded interactions involving Japanese participants, which is notable because much of the existing literature on romantic attraction has been conducted with Western samples. Cultural context matters here: norms around eye contact, smiling, speech tempo, and self-disclosure can differ substantially between Japanese conversational settings and those documented in earlier Western speed dating research. By working with a Japanese corpus, the researchers add a data point from a population that has been underrepresented in computational studies of courtship behavior, and their findings may not map one-to-one onto results from other cultural settings.

Each ten-minute conversation was divided into ten one-minute intervals, and the researchers extracted four broad families of features from each interval. Visual features included facial action units, which are standardized descriptors of visible muscle movements such as brow lowering, smiling, and head motion. Audio features captured properties of the speech signal itself, such as acoustic dynamics, without requiring the content to be understood. Linguistic features were derived from what participants actually said, including semantic representations of their utterances. Finally, interpersonal-interaction features described how the two partners behaved in relation to each other, capturing coordination and mutual patterns that no single individual’s behavior could convey on its own.

The choice of Random Forest classifiers reflects a deliberate trade-off between predictive power and interpretability. Random Forests are ensembles of decision trees that can handle heterogeneous feature types and capture nonlinear relationships, but unlike deep neural networks they can be interrogated with tools such as SHAP, which attributes each prediction to the contributions of individual features. SHAP values quantify how much each feature pushes a given prediction toward or away from a decision to exchange contact information, and aggregating these values across participants and intervals yields a ranking of which behaviors the models relied on most. The researchers used SHAP both for feature selection and for the importance analysis that forms the interpretive core of the paper.

The evaluation design deserves particular attention. Rather than splitting data at the level of individual conversations, the researchers used fully nested participant-level cross-validation, meaning that when the model was tested on one set of participants, no data from those same individuals appeared in the training set. This matters because participants differ systematically in their baseline behavior, expressiveness, and decision tendencies; allowing the same person’s data into both training and test sets would leak information and inflate apparent performance. Nested cross-validation additionally means that hyperparameter tuning occurred within inner loops, keeping the outer evaluation honest. This is a methodologically conservative approach, and it likely explains why the reported performance gains over baseline are modest.

Indeed, the statistical picture is one of suggestive but unconfirmed signals. Across all eight evaluated configurations, the Random Forest models scored numerically higher than a mean class-proportion random baseline, which is what one would expect if the behavioral features carried some signal. However, every 95 percent interval for the difference between model performance and baseline included zero, meaning that none of the individual comparisons reached conventional statistical significance. The best-performing configuration for female participants’ decisions, which used all feature types combined, reached a weighted F1 of 0.6248, while the best configuration for male participants’ decisions, which used visual features alone, reached 0.5379. For context, a weighted F1 of 0.5 would be roughly consistent with predicting the majority class in a balanced problem, so the models appear to extract some information from the conversations, but far from enough to make reliable individual-level predictions.

The asymmetry between the two decision groups is one of the more intriguing results. Features related to female participants’ AU04, the facial action unit corresponding to brow lowering, ranked among the most important predictors in both the all-feature model for female decisions and the visual-feature model for male decisions. Brow lowering can accompany frowning, concentration, or expressions of concern, and its prominence suggests that a female participant’s facial expressions carried information relevant to both her own decision and her partner’s. Prior work on gender differences in mate selection has suggested that women and men may weigh different cues when forming interest, and the finding that different feature configurations performed best for each group is consistent with that literature, even if the present data cannot confirm the underlying mechanism.

The temporal analysis adds another layer. When the researchers examined SHAP importance by one-minute interval, they found that the share of feature importance peaked at six to seven minutes into the conversation for predicting female participants’ decisions and at seven to eight minutes for male participants’ decisions. In other words, the middle-to-late portion of the ten-minute interaction, rather than the opening moments, contributed most to the models’ predictions. This timing is plausible from a psychological standpoint: early minutes of a first conversation are often spent on pleasantries and orientation, while impressions that inform a decision about further contact may consolidate only after several minutes of exchange. It also suggests that studies treating an entire conversation as a single undifferentiated unit may be averaging away meaningful temporal structure.

The authors are careful to note a significant limitation: feature rankings varied across evaluations that used different held-out participants. This instability means that the specific features identified as important, including the prominence of AU04 and the mid-conversation timing peak, should be treated as hypotheses rather than established findings. Small samples, individual idiosyncrasies, and event-specific circumstances can all produce feature-importance patterns that fail to replicate in new data. The researchers explicitly call for independent datasets to determine whether the observed patterns remain stable across samples, a stance that reflects growing awareness in the machine learning community that feature-importance analyses require replication before they can support strong claims.

The study also connects to a practical motivation. A recent Japanese governmental survey documented public demand for support with communication methods, relationship development, and counseling, and the researchers frame their work as a step toward identifying observable behavioral patterns that could eventually inform such support. If future research confirms which conversational behaviors are associated with mutual interest, that knowledge could feed into communication training, dating support services, or tools that help people understand how they come across in first encounters. The authors caution, implicitly, that such applications remain distant given the current state of the evidence.

Methodologically, the work sits within a broader movement in computational social science toward interpretable multimodal analysis of dyadic interaction. Related research in negotiation and job interview settings has shown that mutual non-verbal behavior, considered jointly rather than one participant at a time, can predict immediate reactions and social evaluations. The speed dating paradigm complements this line of work by offering a structured setting with a clear, consequential outcome: contact exchange occurs only when both parties express interest, making it a natural laboratory for studying the behavioral dynamics of mutual choice. By segmenting conversations into fine-grained time intervals and comparing feature sets across modalities, this study offers a template for how future studies might localize when, and through which channels, outcome-relevant behavior emerges in face-to-face interaction.

Subject of Research: Machine learning prediction of speed dating contact-exchange decisions from multimodal behavioral features

Article Title: Predicting female and male participants' contact-exchange decisions from time-segmented multimodal behavioral features in speed dating

Article References: Ogushi, A., Azuma, N., Ishii, R., & Miyata, A. (2026). Predicting female and male participants' contact-exchange decisions from time-segmented multimodal behavioral features in speed dating. Frontiers in Computer Science, 8, Article 1893669. https://doi.org/10.3389/fcomp.2026.1893669

Image Credits: AI Generated

DOI: 10.3389/fcomp.2026.1893669

Keywords: speed dating, machine learning, multimodal behavior analysis, random forest, SHAP, facial action units, nonverbal communication, romantic attraction, contact exchange, cross-validation, human-computer interaction, behavioral prediction

The motivation behind the work lies in a long-standing gap between what language-learning software can measure and what actually happens in a learner’s mind and body. Most computer-assisted pronunciation and grammar tools evaluate linguistic output alone: phoneme accuracy, error counts, fluency scores. Yet second-language acquisition is a deeply embodied process. Speech rate fluctuates under stress, hesitation patterns reveal working-memory strain, gestures and posture shifts signal engagement or confusion, and emotions such as anxiety and enjoyment measurably influence fluency and persistence. A system that hears only the words misses most of the story. Tang’s framework is built on the premise that linguistic output, bodily behavior, and affective responses form a single, intertwined signal that must be modeled together.

Technically, ACG-CMT begins with three parallel encoding branches. The speech encoder does not receive polished pronunciation scores; instead it learns from raw acoustic-prosodic material—Mel-spectral patterns, energy contours, pitch and prosodic rhythm, and temporal features such as pauses—using a Mel-spectrum transform, one-dimensional convolutions, and bidirectional gated recurrent units. The gesture branch applies a spatiotemporal graph convolutional network to body-keypoint trajectories extracted from video, treating joints as graph nodes and their connections as edges, so that gesture amplitude, posture stability, and movement continuity become learnable features. The affective encoder combines facial action units captured by OpenFace with vocal-emotion cues, mapping both into a shared emotional latent space. Crucially, the study enforces a strict separation between predictor inputs and the variables used to construct supervision labels, reducing the risk that the model simply reconstructs its own targets.

Because speech, motion, and facial signals arrive at different sampling rates and evolve on different timescales, the framework next performs temporal alignment. All three encoded streams are resampled onto a fixed-length time window with positional encoding, preserving each modality’s internal order while placing corresponding behaviors on the same temporal axis. A cross-modal transformer then applies attention between modalities: the speech stream can query gesture and affect streams and vice versa, allowing a pause in speech to be linked with a simultaneous drop in gesture activity or a spike in negative vocal affect. Attention analysis of the trained model revealed non-uniform patterns—speech-driven attention to gesture and affect was locally concentrated, while gesture-to-affect attention spread across wider temporal ranges—suggesting the model captures genuinely structured cross-modal relationships rather than averaging everything together.

The architectural centerpiece is the affective-cognitive gate. Conventional multimodal fusion typically assigns fixed weights to each modality after training, but the importance of speech versus gesture versus emotion changes from moment to moment. When a learner mispronounces a word, acoustic features matter most; when comprehension breaks down, facial and vocal affect carry more diagnostic weight; when engagement collapses, posture and gesture become the key evidence. The gating unit generates normalized weights for the three modalities conditioned on latent affective and cognitive context and on pre-decision task context—all computed from information available at inference time, with no access to ground-truth labels. The resulting fused representation feeds a multi-task decision head that simultaneously performs learning-state recognition, affective-cognitive load estimation, and feedback-policy matching.

The empirical evaluation involved 96 non-English-major undergraduates aged 18 to 23, each with at least six years of English study. In a controlled indoor setting, a headset microphone recorded speech at 16 kHz while a front-facing camera captured video at 30 frames per second; interaction logs recorded task types, response times, and feedback events. From 3,840 task segments, the researchers generated 12,864 synchronized five-second analysis windows with 50 percent overlap. Human annotators labeled five learning states—normal, pronunciation deviation, hesitation, comprehension difficulty, and low participation—with strong inter-rater agreement (Cohen’s kappa of 0.824). English teachers independently assigned feedback labels across five categories: error correction, prompting, pacing adjustment, difficulty adjustment, and emotional support. Data were split at the learner level into training, validation, and test sets in a 7:1:2 ratio to prevent overlapping windows from leaking between sets.

The results are striking. ACG-CMT achieved 0.892 accuracy, 0.884 macro-F1, 0.941 AUC, and 0.877 difficult-state F1 for learning-state recognition, outperforming traditional machine-learning baselines such as XGBoost (0.768 accuracy), single-modal deep models, and even recent multimodal transformers (0.876 accuracy for the strongest competitor). For the operational affective-cognitive load reference index—a composite built from NASA-TLX workload ratings, reaction time, pause ratio, and task difficulty—the model reached an R-squared of 0.836 with RMSE of 0.084 and MAE of 0.068. Feedback-policy matching hit 0.861 with an average latency of just 438 milliseconds, fast enough for genuinely interactive tutoring. Ablation experiments underscored the design: removing cross-modal attention cost the most performance (macro-F1 dropping to 0.801), followed by removing the affective-cognitive gate (0.812), while removing the feedback module mainly degraded policy matching rather than state recognition.

Perhaps the most intriguing findings concern interpretability. The gating weights shifted systematically across learning states: under normal expression the three modalities stayed balanced between roughly 0.32 and 0.35; pronunciation-deviation samples shifted weight toward speech; comprehension-difficulty samples shifted toward affect; and low-engagement samples leaned on gesture and emotional information. Precursor signals also emerged before feedback events: in the five seconds preceding a feedback trigger, the pause ratio climbed from 0.214 to 0.414 and the load-risk estimate rose from 0.341 to 0.721. Feedback delivered within a 300-to-500-millisecond window produced the highest state-recovery gain (0.231), compared with 0.148 for faster responses and a meager 0.079 when feedback arrived after 900 milliseconds—evidence that timing, not just content, shapes the effectiveness of adaptive tutoring.

Tang is careful about what these patterns do and do not prove. The gating trajectories and attention maps are described as model-level associations, not causal explanations of human cognition, and the composite load index is explicitly framed as an operational reference rather than a validated physiological measure of cognitive load—no neurophysiological signals such as pupillometry were collected. The overlapping windows mean the 12,864 samples are not statistically independent, and the participant pool of Chinese undergraduates limits generalizability. Still, the study marks a meaningful shift from outcome-oriented assessment toward process-level perception: a tutoring system that reads the whole learner, not just the transcript. Future work, the author notes, should test the framework with more diverse populations, in open-ended and longitudinal learning environments, and against independent physiological measures—steps that will determine whether emotionally and bodily aware AI tutors can move from the laboratory into the classroom.

Subject of Research: Embodied multimodal human-computer interaction using speech, gesture, and affective signals for adaptive English language learning

Article Title: Embodied multimodal human-computer interaction for English language learning: integrating speech, gestures, emotion, and adaptive feedback

Article References: Tang, Y. (2026). Embodied multimodal human-computer interaction for English language learning: integrating speech, gestures, emotion, and adaptive feedback. Frontiers in Neurorobotics, 20, Article 1920739. https://doi.org/10.3389/fnbot.2026.1920739

Image Credits: AI Generated

DOI: 10.3389/fnbot.2026.1920739

Keywords: multimodal AI, affective computing, cross-modal transformer, English language learning, embodied cognition, adaptive feedback, cognitive load, gesture recognition, speech prosody, learning-state recognition, human-computer interaction, neurorobotics

The starting point is an uncomfortable observation about modern digital environments. Interfaces, recommendation systems, and social media feeds have evolved from neutral toolsets into highly optimized, adversarial ecosystems. Deceptive patterns in interface design trick users into unintended actions; algorithmic profiling quietly assembles intimate portraits from innocuous data points; information disorder pollutes the news ecosystem with misinformation, disinformation, and malinformation. Each phenomenon has matured into its own research sub-discipline with specialized taxonomies and countermeasures, but the authors argue that studying them in isolation obscures the structural machinery they share.

That shared machinery, according to the new paper, is the exploitation of well-documented cognitive vulnerabilities. Deceptive patterns capitalize on the fast, automatic, unconscious mode of thinking that psychologists call System 1, discouraging the slower, more reflective System 2. Algorithmic profiling exploits bounded rationality—our inability to weigh abstract, long-term privacy risks against immediate gratification. Information disorder feeds on what researchers term analytical laziness, the tendency to share content without critically evaluating it, compounded by a human bias toward novelty that makes false news spread faster than the truth. In each case, the result is what the authors call a Knowledge-Behavior Gap: a strategically induced rift between what users reflectively know and how they actually behave in the moment.

The evidence for this gap is not new, but the paper’s synthesis draws the contradictions together in a striking way. Users are often aware of manipulative interface design yet comply with it anyway—a phenomenon the authors label manipulation awareness versus behavioral compliance. They express strong privacy concerns yet readily disclose personal data, the well-documented privacy paradox. They value truth and know they should verify what they read, yet actively share unverified, fallacious news. Crucially, the authors emphasize that these contradictions are properties of the digital harms themselves, established in prior literature, rather than measurements from their own tools. The insufficiency of reflective knowledge alone is precisely what motivates a proactive, rather than purely informational, response.

To build that response, the researchers first compiled an annotated portfolio of three mitigation tools from their prior applied research, each targeting a different harm. Trickery is a first-person serious game that casts the player as a new employee in a laboratory guided by an unreliable narrator, navigating seven rooms that each operationalize a real deceptive pattern—from insistent questioning modeled on the nagging pattern to a vending machine that sneaks unwanted items into the player’s cart. The design draws on emotional learning and inoculation theory, the idea that pre-exposure to a weakened form of an argument builds resistance against stronger future attacks. PERSONÆR is a browser extension that runs in the background of a browsing session, extracts keywords from visited websites, and uses a large language model to synthesize a structured 300-word persona—pseudonym, gender, age range, location, and narrative background—simulating how third-party trackers might interpret the user’s passive digital behavior. FallacyCheck, also a browser extension, identifies thirteen types of logical fallacies in news articles and poses non-leading questions in contextual tooltips, prompting users to verify a claim’s logic themselves rather than triggering the defensive backfire effects associated with direct corrections.

When the authors annotated these tools side by side, structural similarities emerged across the research silos. Each tool deploys a persuasive strategy rather than a passive warning: Trickery uses simulation grounded in inoculation theory, PERSONÆR uses reduction executed through narrative framing, and FallacyCheck uses contextual reminders operationalized as non-leading questioning. Each also occupies a distinct intervention mode. Trickery builds capacity outside the moment of vulnerability, serving as restorative training before real-world harm is encountered. PERSONÆR and FallacyCheck provide scaffolding during use, compensating for bounded rationality and analytical laziness precisely at the moment of potential exploitation. And in every case, the goal is not to impose a new, system-defined good behavior but to dismantle pre-existing manipulation and restore the user’s autonomy.

From these observations the authors formalize two constructs of intermediate-level knowledge—a term from human-computer interaction research describing insights that sit between specific design instances and grand universal theories. Systemic Digital Exploitation is proposed as a conceptual construct: the strategic design of digital environments to actively exploit human psychological vulnerabilities, bypass rational decision-making, induce a Knowledge-Behavior Gap, and thwart user autonomy. Counteractive Persuasive Technologies is proposed as a bridging concept connecting theory to practice: digital interventions that actively deploy persuasive strategies to counteract SDE, bridging the gap to restore autonomy and support what legal scholars call the right to mental self-determination—the capacity to form thoughts free from covert algorithmic interference, or hypernudging.

The paper is unusually careful about where its central concept ends. SDE, the authors argue, must satisfy four jointly necessary and independently falsifiable conditions. It must be systemic, meaning a standing feature of the environment rather than an attack aimed at selected individuals—a targeted phishing campaign fails this test because it befalls particular people rather than constituting the ordinary experience of a platform’s user base. It must be digital, meaning the environment furnishes the user’s entire evidential basis for the decision; a supermarket placing expensive products at eye level fails here, because the shopper perceives the shelf directly and can check every option on the spot. It must exploit psychology rather than merely constrain capability—printer firmware updates that disable third-party ink cartridges act against users’ interests but foreclose an action outright, leaving no behavioral alternative to decouple from reflective knowledge. And it must distribute harm asymmetrically without a transparently disclosed, evaluable exchange; a retail loyalty stamp card exploits the goal-gradient effect and loss aversion, but its proposition is stated on its face and can be declined without cost.

This boundary-drawing matters because it sharpens what SDE actually names: not harm, not influence, not adversity as such, but a standing condition of the environment that leaves the user’s capabilities and knowledge intact while routing them past both at the moment of decision. The user is not prevented from acting on their values, nor argued out of them—they are bypassed, with no discrete attack to intercept and no vantage point from which the arrangement could be seen for what it is. It is this specific decoupling, the authors argue, that the four conditions isolate and that counteractive interventions are designed to reverse.

The CPT framework itself raises an obvious ethical paradox: if countermeasures use the same persuasive mechanisms as the systems they fight, do they not risk becoming new manipulations? The authors’ answer is definitional and procedural. A CPT that mandates a specific behavioral outcome is, by definition, not a CPT at all; its sole ethical mandate is restorative, and the final decision remains with the user—even if that autonomously formed choice means accepting a digital risk. The design process enforces this in two steps adapted from the established Persuasive Systems Design model: designers must first identify and name the pre-existing exploitation phenomenon and the psychological mechanism it exploits, fixing the intervention’s persuasive force to an external referent rather than a designer’s goal, and then choose an intervention mode and counteractive strategy informed by Fogg’s Behavior Model and Self-Determination Theory. The PERSONÆR tool illustrates the principle: rather than blocking trackers or warning users away from certain sites, it simply shows them what their algorithmic profile might look like, leaving the decision entirely to them. The authors acknowledge the framework cannot guarantee designer fidelity—a practitioner can name an exploitation pro forma and quietly pursue an agenda—but the claim to be a CPT is falsifiable on inspection. The paper presents no new empirical data, and the authors call for analytical validation through systematic review of independently developed tools and generative studies measuring whether such interventions measurably narrow the Knowledge-Behavior Gap they are theorized to bridge.

Subject of Research: A theoretical framework uniting digital harms such as deceptive patterns, algorithmic profiling, and information disorder under the concept of Systemic Digital Exploitation and proposing Counteractive Persuasive Technologies to restore user autonomy

Article Title: Systemic Digital Exploitation and Counteractive Persuasive Technologies: intermediate-level concept synthesis from an annotated portfolio

Article References: Kronhardt, K., & Gerken, J. (2026). Systemic Digital Exploitation and Counteractive Persuasive Technologies: intermediate-level concept synthesis from an annotated portfolio. Frontiers in Artificial Intelligence, 9, Article 1952265. https://doi.org/10.3389/frai.2026.1952265

Image Credits: AI Generated

DOI: 10.3389/frai.2026.1952265

Keywords: dark patterns, deceptive patterns, algorithmic profiling, misinformation, persuasive technology, user autonomy, mental self-determination, behavior change, human-computer interaction, self-determination theory, digital ethics, Knowledge-Behavior Gap

The research team, led by Maggie Liao, an assistant professor at the University of Georgia who completed her doctoral work at Penn State, and co-author S. Shyam Sundar, Evan Pugh University Professor and James P. Jimirro Professor of Media Effects at Penn State, set out to isolate a specific variable: the degree to which a chatbot communicates conversationally. Both researchers have conducted several prior studies on how people use and trust AI chatbots, and they were well aware that the technology can produce inaccurate information. What they wanted to know was whether the conversational delivery itself, independent of the content, makes users more likely to accept false claims as true.

“We tested whether the conversational nature of GenAI chatbots is driving users to trust misinformation coming from them,” Liao said. “And if that is the case, the natural next question is: What can we do about it?” The question has taken on considerable urgency as chatbots become a mainstream source of information on topics ranging from personal finance to health. Recent survey data cited by the researchers indicate that Americans are increasingly turning to AI chatbots as trusted sources, including for medical questions, at the same time that these systems are becoming more human-like in the way they communicate. The combination, the researchers suggest, creates a potentially problematic dynamic in which the medium’s style overwhelms users’ critical judgment.

The theoretical backbone of the study is a concept the researchers call the “negative machine heuristic,” the ingrained tendency to believe that machines lack the human touch when it comes to intuition and subjective decision-making. Historically, that heuristic has served as a protective filter: people have tended to discount what machines tell them precisely because machines were perceived as mechanical, impersonal, and lacking judgment. The researchers found that the more conversational or personable a chatbot is, the more users curb this negative machine heuristic, effectively lowering their psychological defenses. Strikingly, this happened even when the information being delivered was, in the researchers’ words, “ridiculously wrong.”

Part of what makes conversational AI so persuasive, Sundar explained, is the specificity of its delivery. “The conversational delivery of information by generative AI is hyper-specific to your particular question, so it seems very tailored,” he said. “This is the first time in the history of technology where a machine can actually hold forth a conversation that has not been predicted by a programmer or has not been pre-scripted.” Unlike earlier information technologies, where scripted responses or pre-programmed menus reminded users that a machine was on the other end, generative AI produces spontaneous, unscripted dialogue. That spontaneity signals human-like agency, and users appear to transfer the trust they would extend to a knowledgeable human interlocutor onto the machine.

To test these ideas empirically, the researchers conducted an online experiment with 477 participants. Each participant was told they were helping to test a new AI health assistant and was asked to submit one prompt from a pool of sample prompts to the chatbot. During the interaction, the AI deliberately provided incorrect information. Participants were then asked to indicate how credible they believed the chatbot was. The experiment varied how conversational the AI was, with some participants interacting with a low-conversational version and others with a high-conversational version. The design allowed the researchers to measure whether conversational style alone shifted credibility judgments, holding the accuracy of the information constant at deliberately wrong.

The results were sobering. Participants who received highly conversational AI responses were more likely to find the information credible than those interacting with less conversational versions, and this effect persisted even when the information was blatantly inaccurate. “Some of the misinformation we used in the study was ridiculous,” Liao said. “One example was that ginger can be more effective than chemotherapy in treating cancer. It sounds ridiculous, but if it is given by a very conversational AI, people trust it more. I think that’s worrisome.” The finding suggests that the stylistic polish of a chatbot’s delivery can override even basic common-sense scrutiny, at least in the context of a health-related interaction where users might otherwise be expected to be cautious.

The second part of the study tested a potential remedy. The researchers designed a verification option modeled after the standard human fact-checking process, giving users the opportunity to confirm whether the AI’s responses were backed by evidence. The idea was to counter a common assumption among users that AI is always more objective and accurate than humans, an assumption that can make people suspend the skepticism they would naturally apply to a human source. The experiment compared four conditions: a simple cue consisting of a small icon warning that information may be incorrect; a required check with a button that users had to click in order to proceed; an optional check with a button that users could click if they chose to verify the information; and no verification option at all.

The findings on verification were nuanced and, in some ways, counterintuitive. Users who had the ability to verify the information were more skeptical of the AI’s responses overall, but a passive verification cue alone was not enough to change minds. Trust in the AI system only decreased for users who actively clicked on the verification buttons. In other words, the act of verification itself, rather than the mere presence of a warning icon, appears to be what rebuilds skepticism. When given the option, nearly half of the participants chose to verify the information they received, and those who did were less likely to accept false claims as accurate. The behavioral engagement seems to matter: clicking a button and checking evidence forces a moment of critical evaluation that a passive icon does not.

“The verification option is a useful antidote to the conversational persuasiveness of GenAI,” Sundar said. “That’s what we say in the title of our study: ‘Chat, but verify.’ Enjoy the conversation by all means, but don’t be lulled into believing all that you’re getting.” The researchers hope the results serve both as a warning and as a practical roadmap for the AI industry. As Sundar put it, research showing that verification tools help combat misinformation should persuade AI platforms to take such mechanisms seriously and to develop more checks on information generated by AI. For a technology whose persuasive power grows with every conversational improvement, the study suggests that the solution lies not in making chatbots less engaging, but in building active, user-driven verification into the conversation itself, so that enjoying the chat no longer means surrendering judgment to the machine.

Subject of Research: How conversational AI chatbots increase user trust in misinformation and whether verification tools reduce that effect

Article Title: Chatty chatbots can mislead users, but verification tools help, researchers find

Article References: Chatty chatbots can mislead users, but verification tools help, researchers find. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: AI chatbots, misinformation, generative AI, verification tools, human-computer interaction, media effects, health information, credibility, Penn State, Journal of Computer-Mediated Communication, negative machine heuristic, fact-checking

The research, published in Frontiers in the Internet of Things by Börge Kordts, Lea C. Brandl and Andreas Schrader of the Institute of Telematics, tackles one of the most stubborn problems in pervasive computing. The Internet of Things promises environments filled with sensors, actuators and interfaces that quietly extend our abilities, but the sheer variety of devices and services makes manual integration complex and tedious. Self-organization has long been proposed as the remedy: devices describe themselves, discover each other and form functional ensembles without a technician in the loop. The catch is that when a room reconfigures itself dynamically, users often have no idea how to interact with the result. The system may work perfectly, yet remain a mystery to the very people it is meant to help.

To bridge that gap, the team extends a concept they call Ambient Applications. Unlike ordinary apps, which live on a rectangular screen and have little connection to the physical world, ambient applications are software components defined by their close relationship to a smart environment. They are either controlled by components of the environment or provide output that controls other ambient applications or smart objects. The hardware they run on is, in principle, interchangeable. This opens the door to decoupling interaction from application logic: input devices, output devices and the software in between can be combined in different ways depending on the situation, the user and the resources available. Interfaces need not be graphical at all; projections onto objects in the room or voice-based interaction are equally conceivable.

The technical foundation is the Ambient Reflection framework, which the authors extended for this study. The framework already consisted of three core components. A Smart Object Library lets developers integrate smart objects directly, handling network communication over an event bus. A Virtual Device Daemon, together with its API, acts as a web-socket-based bridge for software that cannot use the library, such as web applications running in restricted environments. And a Description Mediator connects smart objects and applications based on their self-descriptions, using a probabilistic brute-force algorithm as a best-effort solution to the ensembling problem. Every component describes itself in a human- and machine-readable language, covering its input and output interfaces and basic behavior. Applications are deliberately modeled as black boxes: the descriptions explain how to control them, not their internal logic.

The novel contribution is an application management layer bolted onto this architecture. Devices that can host applications run a manager that describes itself and the restrictions of its hardware. When an application is requested, the manager pulls the specified version from a trusted repository, verifies its integrity with a checksum comparison, extracts the archive and launches it with the appropriate runtime environment and configuration. Because some resources, such as a display or audio output, cannot be shared, the team introduced a Limited Application Manager built around a clever device upgrade scheme. A device starts as a minimal idle instance whose only job is to be discoverable and to accept launch requests. When an application arrives, the idle instance shuts down to free its resources, the new application starts as a subprocess, and once the application terminates the idle instance reboots to await the next request. This prevents resource deadlocks on constrained hardware.

Distribution is handled by the Description Mediator, whose REST interface was extended to accept application launch requests. A distribution algorithm matches requested applications to discovered devices based on their capabilities, both hardware and software. Software capabilities are expressed using MIME types, the standard format for indicating media types, while the authors propose a MIME-inspired format for describing hardware properties, including connected peripherals. An application that needs a graphical interface will only be dispatched to a device that can actually display one. After launch and connection, the mediator merges the self-descriptions of all components in the ensemble into an aggregated description, which serves as the raw material for generating user instructions.

That instruction generation is where the self-explainability comes in. An HTML-based rendering engine produces multimedia guides that combine textual descriptions with graphical representations of the required input actions and the corresponding system responses, including animations of the interaction steps. For multi-step tasks, an overview page links to detailed explanations of each task, with navigation elements at the top and bottom. The result is a manual that did not exist until the moment the ensemble formed, tailored precisely to the devices and applications actually present.

To find out whether such automatically generated instructions actually work, the researchers ran a quasi-experimental laboratory study with twelve students from the University of Lübeck, seven male and five female, with an average age of 25 years. Participants completed five tasks of increasing difficulty across three ensembles drawn from representative domains: a smart home scenario and two medical scenarios. They used a ball-shaped interaction device called BIRDY to control an augmentative and alternative communication application, a camera-based Leap Motion hand gesture controller to operate a room-control application with Philips Hue smart lights and a FRITZ!DECT smart thermostat, and finally a finger-worn LITHO touch controller plus a Flic smart button to drive a pain documentation application intended for hospital wards. Instructions were displayed on a separate smart whiteboard, and a wizard-of-Oz setup meant the applications were remotely controlled by the investigator while participants believed they were interacting with the system.

The results were strikingly positive. Every participant solved every task successfully, and all but one identified the correct devices on the first attempt. Questionnaire responses rated the self-explainability of the environment as very well intelligible, with particular praise for the assistance during the learning process. Interviews reinforced this: five participants explicitly found the instructions and their structure clear, and the illustrations and animations drew positive mentions. Yet the study also surfaced instructive failures. Several participants misinterpreted the still images illustrating Leap Motion gestures and the Flic button, and some missed that the LITHO controller required placing the thumb on the device while swiping. Four participants admitted their mistakes came from skimming the instructions rather than reading them carefully. Notably, no consensus emerged on whether animations, illustrations or text were most helpful; all three formats were rated helpful, suggesting that different learning styles demand different modalities. On placement, eleven participants could imagine instructions on a separate device and ten favored integration into the application itself, with a telling split: separate displays were preferred in medical contexts, while integrated guidance seemed suitable for smart homes.

The authors are candid about the limitations. The sample consisted entirely of digitally fluent students, the wizard-of-Oz setup left system autonomy untested, and the evaluation relied on fairly traditional applications displayed on conventional monitors. Still, the core finding stands: given good self-descriptions, the framework can generate comprehensible instructions for even complex, dynamically assembled ensembles, and it answers the question of where self-explaining ambient applications come from and how they enter an environment. The team sees this as a first step toward a genuine application store for smart spaces, where users could publish, install, update and remove self-explaining applications as easily as they install apps on a phone. Future work includes factoring device positions and network topology into the distribution algorithm, deploying the system at scale in real-world settings such as teaching scenarios and ambient serious games, and exploring non-visual instruction formats, from audio guides to augmented reality assistance. If the vision holds, the smart environments of the future may not only run themselves but also teach us, politely and on demand, how to use them.

Subject of Research: Self-explaining ambient application management in smart IoT environments

Article Title: Managing self-explaining ambient applications

Article References: Kordts, B., Brandl, L. C., & Schrader, A. (2025). Managing self-explaining ambient applications. Frontiers in the Internet of Things, 4, Article 1623733. https://doi.org/10.3389/friot.2025.1623733

Image Credits: AI Generated

DOI: 10.3389/friot.2025.1623733

Keywords: Internet of Things, pervasive computing, smart environments, ambient applications, self-explainability, self-organization, service composition, application management, user study, human-computer interaction, smart objects, Ambient Reflection framework

Vibrotactile stimulation, the approach adopted in the study described above, works through an entirely different but complementary pathway. Vibration engages rapidly adapting mechanoreceptors in the glabrous and hairy skin, which are excellent at encoding onset, frequency, rhythm, and spatial location but not naturally tuned to affective qualities. Researchers have therefore learned to map emotional meaning onto vibrotactile parameters: slow, smooth intensity envelopes and low-frequency stimulation tend to be perceived as gentle, calming, or affectionate, while sharp onsets, high frequencies, and rapid rhythms convey urgency, excitement, or arousal. This parametric mapping, sometimes called the haptic affect vocabulary, is what allows a grid of small vibration motors on the hand and forearm to approximate gestures such as a pat, a squeeze, a stroke, or a tap, each carrying distinct social intent.

The decision to distribute actuators across both the hand and the forearm reflects psychophysical evidence about tactile spatial acuity. The fingertips and palm are among the most densely innervated regions of the human body, with two-point discrimination thresholds of only a few millimeters, while the forearm supports much coarser localization. By combining high-acuity hand sites with broader forearm coverage, a wearable can deliver both precise cue information and a more diffuse, bodily distributed sensation. Prior research on wearable haptics has shown that users interpret stimulation on different body sites as corresponding to different gesture categories, so the spatial layout of actuators is not merely an engineering convenience but a communicative design variable in its own right.

The choice of twenty-six actuators per person also illustrates a common trade-off in haptic system design. Increasing the number of actuators improves spatial resolution and gesture expressiveness, but each additional motor raises demands on power, wiring, synchronization, and comfort, and risks producing perceptual crowding in which adjacent sensations blur together. Studies of vibrotactile displays have repeatedly found diminishing perceptual returns beyond moderate densities, particularly on body sites with coarse acuity. A mid-density layout, paired with a wireless and device-agnostic control protocol, represents a pragmatic compromise that keeps the wearable light enough for natural gesture while preserving enough channels to differentiate emotion-specific touch patterns.

That emotion-specific gesture patterns persisted across realities is consistent with a substantial literature on the communicative code of touch. Classic studies in which participants conveyed emotions through touch to a hidden partner’s forearm found that strangers, touching only the arm, could decode distinct emotions at rates well above chance. The gestures people spontaneously produce, including stroking for affection, squeezing for sympathy or support, hitting or poking for anger, and shaking for excitement, form a recognizable vocabulary that appears robust across cultures and relationships. The preservation of these patterns in mediated form suggests that the motor programs of social touch are stable enough to survive translation into actuator commands and that receivers can re-read them from vibrotactile patterns with meaningful fidelity.

The finding that direct physical touch still produced the strongest affective alignment is equally informative. Mediated touch necessarily discards force, skin temperature, texture, moisture, and the continuous compliance of a real hand, all of which contribute to the somatic experience of contact. Temporary actuator arrays also compress the continuous spatial field of the skin into discrete points. Researchers in the affective haptics community therefore treat current mediated systems as approximations that capture gesture identity and temporal rhythm while losing much of the co-regulatory physiology of real contact, including the well-documented heart rate and cortisol effects of supportive touch. The gap between physical and mediated performance quantifies how much of social touch depends on passive mechanical and thermal cues that vibrotactile arrays do not reproduce.

The asymmetry between the roles of initiator and receiver deserves particular attention. The observation that the receiver’s reality influenced emotion recognition more strongly than the initiator’s reality aligns with a broader principle in communication science: interpretation is constrained primarily by the perceiver’s perceptual environment. An initiator selecting and executing a gesture operates largely on their own motor intentions and bodily feedback, whereas the receiver must reconstruct emotional meaning from the incoming signal while immersed in whatever visual, auditory, and proprioceptive context their chosen medium provides. In mixed reality, where virtual content is superimposed on the physical world, competing real-world stimuli and partial perceptual anchoring may reshape how strongly a haptic signal is attended, encoded, and attributed emotional meaning.

The complementary strengths observed between VR and MR echo earlier findings on immersion and embodiment. Fully immersive virtual environments tend to sustain engagement and encourage exploratory behavior, which plausibly explains the longer and more varied touch interactions reported for VR users, since a visually coherent synthetic world invites experimentation with avatars and objects. Mixed reality, by contrast, keeps users perceptually tethered to their own bodies and surroundings, and research on body ownership illusions suggests that seeing one’s real hand or arm facilitates the sense that rendered touch belongs to one’s own body. The stronger embodiment and perceived usability in MR are therefore consistent with the idea that aligning virtual stimulation with visible physical anatomy strengthens self-attribution and reduces cognitive overhead.

These results connect to a large body of work on embodiment in virtual environments. Studies pioneered over the past two decades have shown that synchronizing tactile stimulation with what an avatar’s body visually experiences can produce virtual body ownership, in which participants come to regard a synthetic body, or even a body of a different shape, as their own. Haptic feedback is one of the strongest predictors of this illusion, outperforming visual realism alone. Delivering real vibration on the user’s hand and forearm while a virtual partner touches the corresponding avatar location is therefore not an incidental convenience; it is a mechanism that may actively consolidate the avatar as a felt body rather than a observed puppet.

The study’s cross-reality configuration addresses a practical reality of deployed systems: collaborators rarely own identical hardware. Research on asymmetric collaboration in shared virtual environments has shown that device differences produce imbalances in social presence, influence, and task performance, with fully immersed users often dominating shared space and enjoying richer embodiment. Allowing a VR user and an MR user to exchange social touch meaningfully suggests that haptic channels can partially compensate for asymmetries by giving both parties a common, body-grounded signal that does not depend on matched visual immersion. This has implications for inclusive design, since users with limited mobility, limited space, or motion sensitivity can participate through less immersive devices without losing access to affective communication.

The motivation drawn from pandemic-era isolation connects to growing evidence on the consequences of touch deprivation. Prolonged absence of interpersonal touch has been associated with elevated stress, loneliness, weakened social bonding, and degraded emotional regulation, a cluster of effects sometimes labeled touch starvation in the psychology literature. As remote communication became the default for work, education, and healthcare during social distancing, the absence of the tactile channel was widely noted as a qualitative deficit in video-based interaction. Systems that restore even partial tactile bandwidth to remote settings are therefore of interest not only to technologists but to researchers in social neuroscience, mental health, and relationship science.

Applications beyond dyadic communication are plausible and under active investigation. Therapeutic contexts, including anxiety management, grief support, and long-distance family contact, could benefit from mediated affective touch as an adjunct to videoconferencing. Collaborative work domains such as teleoperation, surgical training, and distributed engineering already use haptics for task information, and adding a social channel could support team coordination when visual and auditory channels are saturated. Educational and performance applications, including ensemble music and dance training conducted remotely, represent further cases where timing and contact cues carry information that screens and microphones lose. Each domain imposes its own constraints on latency, comfort, hygiene, and social acceptability that future iterations must address.

Latency is among the most consequential of those constraints. Human touch perception is highly sensitive to the temporal correlation between an observed gesture and the felt sensation; delays of even a few hundred milliseconds can decouple the visual and tactile streams, weakening body ownership and making the touch feel artificial or misattributed. A wireless, device-agnostic communication framework must therefore maintain tight synchronization across heterogeneous hardware, which is a nontrivial systems-engineering requirement. Reliable gesture-to-sensation mapping also requires accurate tracking of hand pose and contact location, since mismatch between where a virtual touch appears to occur and where vibration is delivered degrades the illusion of contact.

Important limitations frame the interpretation of the findings. Vibrotactile stimulation cannot convey force magnitude, temperature, or skin deformation, all of which contribute to natural touch, and perception of vibration varies with age, skin condition, and placement of the wearables, introducing individual differences that a fixed actuator layout may not accommodate. The laboratory study involved prescribed gestures and unfamiliar partners, whereas everyday social touch is embedded in relationships with shared history and negotiated consent, both of which strongly shape interpretation. Long-term effects of repeated mediated touch on bonding, and cultural variation in touch norms, remain open empirical questions that field studies outside the laboratory would need to answer.

Future directions suggested by this line of work include richer actuator modalities such as thermal displays, electrotactile and ultrasound stimulation, and soft robotic devices that can produce genuine skin stretch and pressure, any of which could close part of the gap between mediated and physical contact. Combining haptics with physiological sensing could allow systems to adapt gesture rendering to a user’s arousal state, creating genuinely reciprocal affective loops. As collaborative extended reality matures, the central lesson is that touch should be designed as a communicative medium embedded in shared embodied interaction, not as a decorative notification layer, and that the social meaning of touch can, with careful engineering, survive the passage between heterogeneous realities.

Subject of Research: Virtual bodies, real touch: haptic social communication across VR, MR, and cross-reality

Article Title: Virtual bodies, real touch: haptic social communication across VR, MR, and cross-reality

Article References: Banerjee, P., Chen, V., Lee, J., & Culbertson, H. (2026). Virtual bodies, real touch: haptic social communication across VR, MR, and cross-reality. Frontiers in Virtual Reality, 7, Article 1923348. https://doi.org/10.3389/frvir.2026.1923348

Image Credits: AI Generated

DOI: 10.3389/frvir.2026.1923348

Keywords: Virtual, bodies, real, touch, haptic, social, communication, across, cross-reality, scientific research

The study matters because augmented reality is expanding rapidly across industry, retail, manufacturing, and education, yet systematic reviews have repeatedly noted that user experience design is often overshadowed by hardware capabilities and visualization techniques. Interaction design sits at the center of this problem. In AR, digital content is merged with the physical environment in real time and in spatially accurate ways, so the interface must bridge two worlds. The dominant assumption in the field has been that natural interaction techniques—gestures, gaze, voice, and direct manipulation of virtual objects—suit AR better than the symbolically mediated menus inherited from two-dimensional desktop computing. But direct, controlled comparisons between the two paradigms have been surprisingly rare, with most evidence coming from evaluations of single, isolated systems.

To close that gap, the researchers designed a within-subjects experiment in which every participant experienced both interaction conditions. Thirty adults, recruited from university and community populations and ranging widely in age and prior AR experience, completed a Tactical Combat Casualty Care training simulation delivered on a Meta Quest 3 head-mounted display. The scenario, built on Design Interactive’s AUGMED platform, focused on identifying and treating massive hemorrhage in a battlefield setting—a task chosen precisely because it demands correct execution of a sequence of physical actions under time pressure, the kind of high-stakes procedural training where interaction quality matters most. The tasks were identical across conditions in objectives, sequence, and content, so any differences in outcomes could be attributed to the interaction modality itself.

The two conditions embodied opposite design philosophies. In the natural interaction condition, participants used gesture-based manipulation, gaze-based selection, and voice commands—inputs chosen because each preserves a direct, continuous correspondence between a physical action and its digital effect, such as reaching toward an object to select it. In the WIMP condition, the same tasks were presented through a graphical interface of floating windows, icons, and menus rendered spatially but retaining a two-dimensional, symbolic interaction structure that required indirect manipulation of virtual content. Condition order was counterbalanced across participants, and a priori power analysis confirmed that thirty participants were sufficient to detect moderate effects with paired-samples t-tests at conventional statistical thresholds.

The measurement strategy combined validated subjective instruments with objective behavioral metrics. Participants completed the System Usability Scale after each condition, the NASA Task Load Index to capture mental, physical, temporal, and other dimensions of workload, and the Presence Questionnaire to assess how strongly they felt present in the augmented environment. The Immersive Tendencies Questionnaire was administered beforehand as a baseline measure of each person’s general propensity for immersion. On the objective side, the experimenters recorded task completion time, misclicks and navigation errors, missed procedural steps, corrective actions, requests for clarification, and observed ratings of interaction smoothness and efficiency during task execution.

The results told a nuanced story. Natural interaction produced significantly higher usability scores, averaging 79.83 compared with 72.67 for the WIMP interface, a statistically significant difference with a moderate effect size. A nonparametric sensitivity analysis confirmed this finding was robust. Presence showed an even stronger advantage, with participants reporting a significantly greater sense of being there under natural interaction, an effect in the medium-to-large range. Crucially, the presence advantage held regardless of which condition participants tried first, and it was not explained by individual differences in immersive tendency, which showed no significant correlation with presence in either condition. The interaction technique itself, rather than a participant’s personality, appears to have driven the immersion benefit.

Objective performance measures reinforced the pattern. Participants committed significantly fewer navigation errors, missed fewer procedural steps, required fewer corrective actions, and demonstrated smoother observed interactions under the natural paradigm, with effect sizes ranging from small to medium. Yet several measures showed no significant difference at all: task completion time, requests for clarification, and overall observed efficiency were statistically equivalent between conditions. Cognitive workload, measured by the NASA-TLX, also showed no significant difference between paradigms, despite mean scores trending slightly lower for natural interaction. In short, natural interaction made people more accurate and made the experience feel better, but it did not make them quicker or lighter on mental demand during initial use.

One of the most instructive findings emerged from the exploratory order analysis. Perceived workload was significantly lower during participants’ second experimental session regardless of which interaction condition they were using, with a mean reduction of 5.70 points on the NASA-TLX. Similar order effects appeared for task completion time, clarification requests, and subjective efficiency ratings. The researchers interpret this as a learning or familiarization effect: participants devoted substantial mental effort to understanding the task, the AR environment, and the interaction paradigm during their first exposure, and that burden diminished with experience. Because no significant interaction between condition and order was observed, the effect reflects acclimation to the setup rather than an inherent advantage of either interface—a reminder that in immersive technology research, the novelty of the medium itself can dominate early workload measurements.

Taken together, the findings suggest that the principal value of natural interaction in AR lies in improving the overall quality of the user experience rather than maximizing efficiency. The authors argue this distinction is especially important in domains such as training, maintenance, healthcare, and decision support, where accuracy, user confidence, and sustained engagement matter alongside productivity. Natural interaction appears to encourage deliberate, embodied movements that prioritize accuracy and confidence over raw speed, and because both interfaces in the study were designed to support successful task completion, opportunities to observe large efficiency differences may have been limited. The researchers also acknowledge limitations: the results come from a single training application on one hardware platform, the sample was drawn from university and community populations rather than operational end users, and the study relied on domain-general instruments rather than emerging AR-specific evaluation frameworks that might better capture spatial and embodied dimensions of interaction.

Future work, the authors suggest, should replicate these findings across diverse AR applications and populations, investigate why natural interaction improves usability and presence without reducing perceived workload, and examine how the benefits evolve over long-term use, in collaborative settings, and in more complex operational scenarios. Disentangling the individual contributions of gesture, gaze, and voice would also help designers of next-generation AR interfaces decide where to invest. For now, the message for the field is clear: if the goal is interfaces that are intuitive, immersive, and forgiving of human error, designing interaction techniques that align with people’s natural behaviors offers measurable advantages—provided that speed alone is not the yardstick.

Subject of Research: A controlled comparison of natural versus WIMP-based interaction paradigms in augmented reality combat casualty care training

Article Title: Evaluating interaction methods in augmented reality training

Article References: Hughes, C. L., & Karwowski, W. (2026). Evaluating interaction methods in augmented reality training. Frontiers in Virtual Reality, 7, Article 1939058. https://doi.org/10.3389/frvir.2026.1939058

Image Credits: AI Generated

DOI: 10.3389/frvir.2026.1939058

Keywords: augmented reality, natural interaction, WIMP interface, user experience, presence, cognitive workload, usability, Tactical Combat Casualty Care, Meta Quest 3, training simulation, interaction design, human-computer interaction

The core problem the review addresses is deceptively simple to state. Human users sit at the center of every VR and AR system, and immersive interaction depends on acquiring body movements, postures, and physiological signals in real time. Conventional wearable sensors, however, largely depend on rigid components such as batteries and stiff printed circuit boards. That rigidity brings considerable wearing discomfort and interferes with natural human movement, undermining the very sense of presence these technologies are meant to create. A sensor that fights the body’s motion is a poor partner for a system whose entire purpose is to make the body feel transported elsewhere.

TENG fibers offer a fundamentally different approach. Triboelectric nanogenerators exploit the triboelectric effect, the familiar phenomenon behind static electricity, in which two materials become electrically charged through contact and separation. When engineered into fiber form, these devices convert mechanical energy from the wearer’s own movements directly into electrical signals. The result is a sensor that needs no external power source for its basic operation: the act of moving, pressing, stretching, or breathing generates the signal that reports the movement itself. Dong and his colleagues emphasize that this self-powered operation, combined with high sensitivity and fast response, distinguishes TENG fibers from the piezoresistive, capacitive, and piezoelectric pressure sensors that dominate the current wearable landscape.

The choice of fibers and textiles as the platform is not incidental. As Dong notes, fibers and textiles are essential to the human body and can be integrated conformally without interfering with daily activity. Clothing is the one technology humans already wear for hours at a time, and embedding sensing into its very threads means the interface disappears into something people put on every morning anyway. The review highlights that these fiber- and fabric-based sensors are flexible, breathable, and comfortable, qualities that rigid sensor patches and strapped-on hardware struggle to match. In principle, a TENG-enabled garment can accompany the wearer through an entire day without ever announcing itself as a piece of electronics.

Technically, the review details two principal fiber architectures: single-electrode and dual-electrode designs. In a single-electrode configuration, the fiber’s conductive layer works against an external reference, simplifying wiring and integration into fabrics. Dual-electrode designs enclose the triboelectric pair more completely, which can improve signal robustness and control. Each architecture involves trade-offs in output performance, durability, and manufacturability, and the authors survey how researchers have balanced these factors across a growing body of laboratory demonstrations. The distinction matters in practice, because a fiber that performs well on a benchtop may behave very differently once knitted into a garment, washed, folded, and worn against moving skin.

Equally important is the toolbox of scalable fabrication routes the review catalogs. The authors describe wet spinning, electrospinning, melt extrusion, 3D printing, rope braiding, thermal drawing, and dip-coating as methods for producing TENG fibers and integrating them into textiles. These techniques span the range from laboratory precision to industrial throughput. Electrospinning, for instance, can draw fibers with fine microstructural control that enhances surface contact and thus electrical output, while melt extrusion and thermal drawing point toward continuous, high-volume production. Rope braiding and dip-coating offer paths to combine functional fibers with conventional yarns, embedding sensing capability into ordinary-looking fabric. The breadth of this toolbox is a signal that the field is moving beyond one-off prototypes toward manufacturable devices.

Boosting electrical output remains a central theme, since the amplitude and stability of the triboelectric signal determine how reliably a system can distinguish one gesture from another. The review surveys strategies to raise output, from material selection and surface engineering to structural design of the fibers themselves. Higher output translates directly into better signal-to-noise performance in real-world conditions, where a sensor woven into a sleeve must pick out the press of a finger against the background noise of an arm swinging, fabric rubbing, and ambient electromagnetic interference. The authors also stress the need to reduce crosstalk between fibers, a challenge that grows as more sensing elements are packed into a single garment, and to minimize environmental interference from temperature and humidity, both of which can alter triboelectric charge transfer.

The demonstrated applications give the technology a concrete, and occasionally playful, face. According to Dong, an advantage of TENG fibers is that they fuse sensing into a single soft, wearable device. Woven into garments, they act as virtual buttons and foldable keyboards, turning a sleeve or a shirt front into an input surface for a headset wearer who cannot see a physical keyboard. They can visualize sports motions and vital signs such as pulse and breathing, feeding real-time physiological data into training or healthcare applications. They can recognize hand gestures, including a sign-language-to-speech system that achieved 98.63 percent accuracy, a figure that hints at genuine assistive potential beyond gaming. And they can control virtual characters in games through wristbands, smart socks, and self-powered dance mats, letting the whole body serve as a controller.

That last set of examples illustrates why the technology fits VR and AR so naturally. A dance mat that powers itself from the dancer’s own footsteps needs no batteries to swap and no cables to tangle. Smart socks that register foot pressure can translate gait and balance into avatar movement. A wristband that senses the subtle flexions of the hand can drive fine-grained manipulation of virtual objects. In each case, the energy that would otherwise be wasted as mechanical motion is harvested and repurposed as the data signal itself, an elegant convergence of power management and sensing that conventional sensor architectures cannot replicate without added hardware.

Looking forward, the authors frame the remaining challenges as three improvements and three reductions: improve wearing comfort through lighter and thinner fibers, raise electrical output, and broaden real-world applications, while reducing crosstalk between fibers, minimizing environmental interference, and lowering manufacturing cost. Overcoming these hurdles, Dong says, will be essential to move TENG fibers from laboratory demonstrations into comfortable, reliable, and practical VR and AR products. The review thus serves as both a progress report and a roadmap. The physics of contact electrification is well understood, the fabrication methods are proliferating, and the application demos are compelling. What remains is the unglamorous engineering of scale: fibers that survive washing, signals that survive humidity, and production lines that survive cost scrutiny. If those problems fall, the next generation of immersive interfaces may not be strapped onto the body at all, but woven into it, sensing every gesture while drawing its power from the gestures themselves.

Subject of Research: Triboelectric nanogenerator fibers for self-powered sensing in virtual and augmented reality

Article Title: Weaving the future of virtual and augmented reality with triboelectric fibers for self-powered sensing

Article References: Weaving the future of virtual and augmented reality with triboelectric fibers for self-powered sensing. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: triboelectric nanogenerators, TENG fibers, virtual reality, augmented reality, wearable sensors, self-powered sensing, smart textiles, human-computer interaction, gesture recognition, e-textiles, wearable electronics, flexible sensors

The study was led by Porismita Borah, a professor in Washington State University’s Edward R. Murrow College of Communications and corresponding author of the new publication. Her co-authors were Ziyao Zhang, a PhD student at WSU; Xiaohui Cao, a PhD student at the University of Wisconsin-Madison; and Danielle Ka Lai Lee, an assistant professor at Hong Kong Shue Yan University. Their work addresses one of the most stubborn problems in the misinformation literature: corrections often fail, and even when they succeed, it has been unclear why. People who are told that a claim they believe or shared is wrong can feel challenged or insulted, react defensively, or simply doubt the credibility of whoever is doing the correcting.

Borah has spent roughly a decade studying how misinformation spreads and how it might be countered, and her latest publication advances the field’s understanding of how empathy functions in correction messages. Prior research in this area has produced conflicting conclusions, with some studies finding that a warm, understanding tone reduces misperceptions and others finding no such effect. Borah’s team added a crucial new layer to the question by testing the effectiveness of tone against the expectations and beliefs of the person receiving the correction, rather than treating all audiences as interchangeable.

The central concept in that layer is anthropomorphism, the well-documented psychological tendency to assign human characteristics to animals, machines, and other non-human entities. In the context of conversational AI, anthropomorphism describes the degree to which a user perceives an AI system as humanlike rather than as a purely technical tool. The study’s key insight is that the optimal communication strategy differs across this spectrum. For people who view AI strictly as a machine, corrections delivered in a neutral, just-the-facts tone proved most persuasive. For those who believe AI can be humanlike, an empathetic, understanding tone worked best.

What makes the finding striking is what did not matter. In multiple studies, the team found that corrections could work most of the time, but the source of the correction was largely irrelevant to its persuasive power. As Borah explained, it did not necessarily matter whether the correction came from a human being or an AI agent; what mattered was the tone and how that tone aligned with people’s beliefs about whether AI agents should behave more like humans or more like machines. In other words, a mismatch between message style and user expectations could blunt even an accurate correction delivered by a well-designed system.

To test these effects rigorously, the researchers conducted a randomized online experiment with 857 parents of children within the age range recommended to receive the vaccine for human papillomavirus, or HPV. The choice of topic was deliberate. HPV is spread through sexual contact and can cause a range of serious health problems, including several cancers. The HPV vaccine is considered safe and effective, yet it has been the subject of widespread and persistent misinformation, making it an ideal case for studying how false health beliefs form and how they might be corrected.

Participants were first evaluated for their level of anthropomorphism belief, then shown a simulated Facebook comment thread. The thread began with a false claim stating that HPV vaccines increase the risk of neurological problems. An AI corrections account then engaged with the claim directly in the comment thread, and the style of its response varied. The neutral answers used direct, plain language, along the lines of stating that the claim was not true and that scientific studies had shown no link between HPV vaccines and the neurological conditions in question. The empathetic tone was warmer, opening with an acknowledgment of the person’s concern, such as expressing that the researchers heard the individual, before pivoting to the scientific evidence.

The results were clear: the correction was most effective at reducing misperceptions when its tone matched the respondent’s anthropomorphism beliefs. A machinelike user confronted with a just-the-facts correction updated their beliefs; a user inclined to see AI as humanlike responded better to warmth and understanding. The finding adds an important dimension to the technical design of automated fact-checking systems, because it implies that a one-size-fits-all correction strategy will systematically fail for part of any large audience. Matching conversational style to individual expectations of the technology could unlock measurably better outcomes at population scale.

The practical implications reach well beyond academic journals. The authors suggest that social media platforms, government agencies, news organizations, and other institutions that confront health misinformation could target their AI fact-checking agents according to a user’s level of anthropomorphism. One proposed approach is a simple onboarding step for users of platforms or accounts that assesses their anthropomorphic tendencies, allowing the AI’s conversational tone to be adjusted accordingly. Such an intervention would be technically lightweight compared to the far harder task of changing underlying beliefs, and it could be integrated into existing comment-moderation and community-notes systems that already deploy AI to flag or respond to dubious claims.

As Borah notes, the problem of misinformation is critical and it is not going away, and the effectiveness of corrections depends on many factors, including the way accurate information is communicated; an empathetic tone may often work better than a condescending one, while factors such as race and gender also shape how messages land. Her conclusion is a sober reminder for engineers and communicators alike: researchers are ultimately trying to study humans, and humans are remarkably complex. The lesson of this study is not that AI is a silver bullet against falsehood, but that the oldest rule of persuasion, know your audience, applies with full force even when the messenger is a machine.

Subject of Research: How the tone of AI-generated corrections, matched to users' anthropomorphism beliefs, reduces vaccine misinformation

Article Title: AI can help correct medical misinformation — when it uses the right tone

Article References: AI can help correct medical misinformation — when it uses the right tone. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: artificial intelligence, misinformation, health communication, HPV vaccine, anthropomorphism, vaccine misinformation, social media, empathy, fact-checking, persuasion, human-computer interaction, Washington State University