Virtual Reality Games With AI Conversation Could Transform Speech Therapy for Children

Virtual reality has long been pitched as the future of medicine, but one of its most quietly revolutionary applications may be happening not in operating theaters but in speech therapy rooms, where children wearing headsets are learning to talk. A research team from the Bulgarian Academy of Sciences and South-West University Neofit Rilski has developed a complete methodology for building interactive VR serious games for speech and language therapy, together with a multidisciplinary assessment framework designed to measure whether these immersive experiences actually work. The study, published in Frontiers in Virtual Reality, describes six VR games equipped with large language model based conversational AI, tested in preliminary sessions with ten children and adolescents with language disorders aged five to thirteen, alongside ten speech and language therapy students who evaluated the same experiences from a clinician’s perspective.

The motivation behind the work stems from a persistent problem in the growing field of VR rehabilitation: most existing tools are built by developers chasing immersion and visual spectacle rather than by clinicians pursuing therapeutic goals. The researchers point out that many VR systems for speech therapy are misaligned with established intervention frameworks, ignore receptive language skills such as auditory comprehension and semantic mapping, and often exclude speech and language therapists from the design process entirely. Where conventional therapy depends on a human therapist dynamically adjusting language complexity, emotional tone, pacing and prompting in real time, VR environments frequently replace this adaptability with rigid, scripted interactions that can increase cognitive load or social pressure for vulnerable children.

To close this gap, the team adopted a design-based research methodology, in which every iteration of the games was evaluated in real therapy sessions and refined according to observed speech frequency, emotional engagement, willingness to communicate, and signs of comfort or distress. The resulting user experience model rests on five pillars: simplicity of scenes and interactions, clarity of the virtual avatar’s prompts, comfort and safety, therapist control over the session, and continuous monitoring of outcomes through an integrated assessment framework. Crucially, the intervention was deliberately balanced across both expressive and receptive language domains, addressing a documented bias in VR research toward language production at the expense of comprehension.

Technically, the platform is a web-based VR serious game hub built with Node.js, Express.js, Socket.io and Three.js, delivering WebXR content over a secure HTTPS connection and storing session data in MongoDB. The conversational AI pipeline chains three components: browser-based speech-to-text through the Web Speech API, a locally hosted large language model, and Google Text-to-Speech for synthesis. When a child selects an object in the virtual scene and speaks, the system transcribes the utterance, generates a contextually appropriate response constrained by a strict therapeutic prompt, and speaks it back through a virtual avatar of the NAO robot, which acts as a visual mediator of communication and gives the child the sense of a genuine conversational partner. Measured end-to-end response latency averaged 692 milliseconds, fast enough to sustain natural turn-taking.

The six games, including Fruit Bingo, Animal Bingo, Hungry Dinos, The Amazing Seabed, Tell Me a Story and NAO’s Christmas, each target specific therapeutic objectives, from receptive vocabulary and single-keyword comprehension to narrative sequencing, morphosyntax and social-pragmatic confidence. Every parameter can be personalized by the therapist: the number and placement of objects, the timing of spoken instructions, voice gender and speech rate, spawn positions, reward content, and even which controller button a small child presses. In Fruit Bingo, for example, the NAO avatar names objects slowly and clearly, offers positive reinforcement for correct answers, and simply repeats the instruction without negative feedback when a child errs, keeping the experience low-pressure.

Safety received as much attention as engagement. Because VR manufacturers generally recommend headsets only for users aged twelve or older, the team built a structured supervised-usage protocol: dosage scaled by developmental stage, from five to ten minutes per session for five-to-seven-year-olds up to twenty minutes for eleven-to-thirteen-year-olds, at most two to three sessions per week. Environmental safeguards included seated interaction, limited virtual locomotion, reduced motion parallax, fixed horizon references and a stable frame rate above ninety frames per second to minimize sensory conflict. Sessions began with health screening, continued under continuous real-time adult supervision with the therapist watching a live stream of the headset view, and ended with post-session monitoring. The team even developed a child-friendly pictorial cybersickness scale using emoticons. Notably, no symptoms of cybersickness or dizziness were observed in any session.

The assessment framework itself operates at four analytical levels, from a single session’s micro-analysis of reaction times and accuracy, through daily group statistics and per-game longitudinal trends, to a global overview comparing all games and participants. A key proposed metric is speech-to-selection response time, the interval between a spoken instruction and the child’s successful selection of the target object, which the authors argue captures auditory comprehension, lexical-semantic processing, visual search and decision-making in a single psychometrically grounded measure. In parallel, fifty speech recordings were analyzed in Praat by a linguist and a speech and language therapist, who coded six parameters including phonological inaccuracy, morphosyntactic inaccuracy, emotional expression, naming and speech reaction time.

The statistical results, though preliminary, are striking. Friedman tests revealed significant differences across games for total speech duration, morphosyntactic inaccuracy, emotional expression and naming, with Bonferroni-adjusted post hoc comparisons confirming that naming durations were significantly longer in the Bingo games than in Tell Me a Story, consistent with their different communicative demands. The Amazing Seabed game produced the highest mean speech duration at nearly forty-seven seconds and the strongest emotional engagement, while average speech reaction times remained below one second in most games, indicating rapid turn-taking. The analyses suggest the games effectively capture and hold children’s attention, elicit high emotional expressiveness, and motivate communication, something children with communication disorders rarely display spontaneously.

Questionnaire data reinforced the picture. Children rated their experience on a three-point emoji scale and returned means between 2.8 and 3.0 on nearly every item, with perfect scores for enjoying the games, hearing and understanding the NAO robot, and absence of cybersickness. All participants said they wanted to play again, and most preferred VR to a tablet alternative. An interesting developmental split emerged: younger children interacted with controllers, while older participants favored hand gestures. The therapy students, rating the system on a five-point scale, gave an overall mean of 4.69, with the clarity of instructions earning a perfect five, and half of them singled out The Amazing Seabed as their favorite for its immersive realism.

The authors are careful to frame these findings as a feasibility study rather than proof of therapeutic efficacy. The sample of ten children was small and diagnostically heterogeneous, there was no control group, and no pre- and post-intervention comparisons, all of which limit generalizability. Larger samples, longitudinal designs and comparisons with typically developing peers are needed before the framework can claim clinical validity. Still, the study delivers something the field has lacked: a clinically grounded, safety-conscious, fully instrumented blueprint for bringing conversational AI into immersive therapy, one that treats the therapist not as a bystander but as the central controller of a personalized virtual treatment room. If the promising early signals hold, the humble speech therapy session may never look the same again.

Subject of Research: Design and evaluation of virtual reality serious games with conversational AI for speech and language therapy in children with language disorders

Article Title: Designing interactive VR scenarios for speech and language therapy with the assessment framework

Article References: Stefanov, T., Andreeva, A., Padareva-Ilieva, G., & Lekova, A. (2026). Designing interactive VR scenarios for speech and language therapy with the assessment framework. Frontiers in Virtual Reality, 7, Article 1954543. https://doi.org/10.3389/frvir.2026.1954543

Image Credits: AI Generated

DOI: 10.3389/frvir.2026.1954543

Keywords: virtual reality, speech and language therapy, serious games, conversational AI, large language models, language disorders, children, user experience, assessment framework, cybersickness, phonetic analysis, receptive language