The research, scheduled for presentation on October 8 at the IEEE International Symposium on Mixed and Augmented Reality in Bari, Italy, is the first to systematically evaluate hallucination awareness cues for embodied conversational agents. These agents are AI-powered systems that go beyond plain text, presenting themselves as graphical characters capable of gestures, facial expressions, and natural speech. They are rapidly appearing in applications from customer service to education, and they inherit a well-known weakness from the generative AI models that power them: a tendency to produce untrue statements, commonly called hallucinations, delivered with the same fluent confidence as accurate information.

In conventional text-based chatbots, the problem of flagging uncertainty has a fairly mature toolkit. Verification systems can check an agent’s statements against trusted sources, and when something looks unreliable, the interface can highlight the relevant words or attach a source link for the user to examine later. Qiao Jin, corresponding author of the paper and an assistant professor of computer science at NC State, notes that these cues are relatively straightforward in text-based conversational interfaces. The challenge, he explains, is far thornier in immersive environments like VR, where conversational agents appear as human-like characters speaking directly to users, and where spoken responses pass by quickly, leaving no persistent trace to highlight or annotate.

To explore how uncertainty might be communicated in this new context, the researchers recruited 24 college students and had each of them interact with a VR system featuring an embodied conversational agent resembling a woman wearing a suit. Participants encountered multiple hallucinated statements under four different experimental conditions. In the first, the agent used embodied cues, making physical gestures associated with the unreliable statement, such as crossing its arms when it was uncertain. In the second, icon cues appeared next to the agent, notifying users that there was additional information they should check about a statement. In the third, text cues were used: every statement the agent made appeared as a subtitle on the screen, and those subtitles could be highlighted in different colors to signal potential uncertainty. The fourth condition served as a control, with no additional cues of any kind.

After each interaction, the team measured how well participants detected hallucinations and gathered detailed feedback through questionnaires and interviews, probing not only whether the warnings were understood but also how much each technique disrupted the participant’s sense of being present in the virtual world. That second dimension matters enormously, because immersion is the entire point of embodied agents. A warning system that constantly yanks users out of the experience defeats the purpose of building lifelike characters in the first place, as Xiaoran Yang, first author of the paper and a Ph.D. student at NC State, emphasizes: the evaluation looked both at which cues were most effective at alerting users to potential hallucinations and at which techniques best preserved the user’s sense of immersion in the VR experience.

The results revealed a genuine trade-off rather than a single winner. All three cue types outperformed the control condition at helping users identify hallucinations, confirming that some form of signaling is far better than silence. But each approach carried distinct strengths and weaknesses. Embodied cues, delivered through the agent’s own body language, were associated with the highest levels of immersion and the highest trust in the agent itself. Users remained inside the fiction of talking to a person, and the character’s gestures felt like a natural extension of human communication. The cost was interpretability: subtle physical signals were harder for participants to decode reliably, meaning some hallucinations slipped past even attentive users.

Text cues sat at the opposite end of the spectrum. By displaying subtitles that could be color-highlighted, the system gave users the clearest possible interpretation of which statements were questionable, making text cues the most effective technique for actually catching hallucinations. Yet this clarity came at a price. Participants reported that reading subtitles distracted them from everything else happening in the virtual environment, pulling their attention away from the agent, the setting, and the flow of conversation. The very mechanism that made the warnings legible also fragmented the immersive experience that embodied agents are designed to create.

Icon cues emerged as a pragmatic middle ground. A small icon appearing beside the agent conveyed more interpretable information than body language alone, while causing less disruption to immersion than a wall of highlighted subtitles. For developers who need a single, balanced solution, this middle path may prove attractive, offering meaningful gains in hallucination detection without the steep immersion penalty of on-screen text or the ambiguity of purely gestural signals.

The implications extend well beyond the laboratory. Educational programs, training simulations, and virtual assistants are among the most promising applications for embodied conversational agents, and these are precisely the contexts where a confidently stated falsehood can do real harm, teaching a student something incorrect or steering a user toward a bad decision. The study’s findings suggest that designers need not choose blindly among warning strategies. By selecting different cues, or combinations of cues, for different contexts, developers can tune the balance between communicative clarity and experiential continuity, improving a system’s ability to communicate clearly and effectively about potential hallucinations while limiting the impact on the user’s experience, as Jin observes.

There is also an important open problem that the researchers themselves flag: culture. Body language is not a universal language, and a gesture that reads as uncertainty in one cultural context may signal respect, deference, or something else entirely in another. Avoiding eye contact, for example, may indicate doubt in one setting but politeness in another. Before embodied cues can be deployed at scale, additional work will be needed to ensure that the body language of these signals can be understood by users from diverse cultural backgrounds. The paper, titled Signals of AI Hallucination: Designing Hallucination-Aware Cues for Embodied Conversational Agents in VR, was co-authored by Yang Zhan, Yuxuan Huang, Yichen Yu, and Noboru Matsuda of NC State, Xie He of Carnegie Mellon University, and Zhuo Wang of Xi’an Jiaotong-Liverpool University. As generative AI characters step out of the chat box and into our headsets, the question of how they should admit doubt may become one of the defining design challenges of immersive computing.

Subject of Research: Designing hallucination-aware warning cues for embodied conversational AI agents in virtual reality

Article Title: Researchers offer three ways to alert users to AI hallucinations with ‘embodied’ agents

Article References: Researchers offer three ways to alert users to AI hallucinations with ‘embodied’ agents. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: AI hallucinations, embodied conversational agents, virtual reality, human-computer interaction, generative AI, user experience, immersion, verification cues, NC State University, ISMAR, educational technology, uncertainty communication