Research report: AI-driven virtual reality simulation for nursing de-escalation training: development, automated assessment, and pilot expert evaluation

A pilot study of an AI-driven virtual-reality nursing simulator found that its automated scoring system classified 74 of 77 faculty participants’ spoken responses in line with expert reference labels, an agreement rate of 96.10% with a 95% Wilson confidence interval of 89.16% to 98.67%. The system combines an immersive scenario with an angry virtual patient, cloud-based conversational artificial intelligence, speech-to-text processing and section-specific BERT classifiers. Seven nursing faculty members completed the evaluation, but the researchers emphasized that the study tested feasibility, usability and perceived educational value rather than nursing-student learning or clinical effectiveness.

The report describes a development and pilot expert-evaluation study, with no nursing students enrolled and no measurement of whether the simulator improves competence, patient outcomes or human-centered care in clinical practice. The simulated task placed a learner in the role of a nurse attempting to de-escalate John, an angry patient in a clinic. Built in the Unity 3D platform, the experience includes a tutorial, a waiting room and doctor’s office, and a debriefing room.

The learner wears a Meta Quest 3 headset, uses handheld controllers and communicates with the patient by speech. The scenario was inspired by the Canadian Alliance of Nurse Educators Using Simulation’s De-escalation of Angry Patient Assessment Rubric, with the first author and a nursing education expert mapping the rubric onto a sequential storyline. The system is designed for training in verbal de-escalation and therapeutic communication, competencies that the report identifies as important in nursing education but difficult to practise consistently because real encounters with agitated patients are limited by safety and logistical concerns.

The 11-part storyline moves from establishing communication to discussing treatment. Sections 1 through 5 focus on a professional introduction, inviting the patient to a private room, exploring the reason for the visit, acknowledging frustration and possible links between emotional distress and physical symptoms, and validating emotion through active listening. Sections 6 through 8 address direct suicide-risk assessment, redirection toward abdominal assessment and acknowledgment of clinical findings alongside the patient’s emotional state and behavior.

Sections 9 through 11 cover reassurance, inquiry about medications and collaborative discussion of acceptable treatment options while respecting patient autonomy. Each response is assigned to one of three levels: Novice, Intermediate or Competent. The reference classes were defined with a nursing education expert to represent responses that omit key elements, responses that meet the expected competency incompletely, and responses that are clear, empathetic, professional and clinically appropriate.

Convai provides the virtual characters’ conversational capabilities through an application programming interface. Its cloud-based speech-to-text service converts the learner’s speech into text, which is used to generate the patient’s next response, while text-to-speech produces the audible answer. A narrative-design function uses a tree-like structure of objectives and decision paths so that dialogues can vary without losing the required sequence of 11 sections.

The patient’s backstory was adapted from the CAN-SIM angry-patient scenario, and the same narrative constraints and moderation settings remained active during the suicide-risk section to reduce the likelihood of inappropriate or unsafe generated content. The simulation also displays transcripts and instructional prompts, uses custom animations and facial movements, and includes a virtual facilitator in every room to provide guidance. Automated grading operates separately from the conversational layer.

The researchers fine-tuned 11 models from the pretrained bert-base-uncased model, giving each scenario section a classifier tailored to its communication objective. Each model was trained with a dropout rate of 0.3 and a linear classification head with three outputs corresponding to the learner levels. Inputs were tokenized with truncation, fixed-length padding and a maximum length of 64.

Each dataset was divided 80% for training and 20% for validation with random seed 42. Fine-tuning used AdamW with a learning rate of 2 × 10−5, cross-entropy loss, three epochs and batches of eight. The section-specific design was intended to reduce ambiguity by allowing each model to focus on a distinct conversational context, expected intent, therapeutic language and learning objective.

At runtime, responses were sent from Unity to the relevant model through an HTTPS REST API on a separate computer. Because suitable collections of authentic nursing-student responses were unavailable, the researchers generated labeled training data with GPT-4o. Section-specific prompts included the scenario context, dialogue history, target performance level and expected competency.

Additional prompts were used to encourage variation in wording, length, punctuation and communication style. The researchers manually reviewed the generated responses, removed duplicates and near-duplicates, and excluded responses that were clinically implausible, inconsistent with the target competency or unsuitable for the assigned level. A nursing education expert reviewed representative examples and grading criteria with the first author.

Class distributions varied across sections because the objective was sufficient representation and diversity rather than artificially balanced classes. The labels were established through collaborative review and consensus rather than independent coding by multiple assessors, so inter-rater reliability could not be calculated. During the pilot, the models assessed uncorrected speech-to-text transcripts from the faculty members, exposing the pipeline to the recognition errors expected in real-time use.

A request to Convai took about 2.21 seconds under the selected settings, while the general delay between a learner speaking and the virtual patient answering was approximately three to five seconds. Each assessment cycle took about 10 seconds and ran in the background while the simulation and interaction continued. The authors emphasized that these timings were estimated from system use rather than formally benchmarked.

Eight of the 11 sections achieved seven correct classifications out of seven, while Sections 3, 5 and 8 each achieved six of seven. With only seven responses per section, the confidence intervals were wide and overlapping; the researchers therefore treated section-level differences as descriptive rather than statistically meaningful. The three disagreements illustrated the difficulty of scoring context-dependent communication.

In Section 3, the response “Okay John, so you came with abdominal pain, tell me more” was labeled Novice by the model but Intermediate by the expert, with the model assigning similar probabilities to all three classes. In Section 5, the model rated a response about the patient’s abdominal pain as Competent, whereas the expert considered it Novice because it did not validate emotion or address anger. In Section 8, a compassionate question about the patient’s support system was classified as Intermediate by the model but Novice by the expert because it omitted the abdominal-assessment findings and the patient’s agitated behavior.

These cases suggest descriptive weaknesses in recognizing specific emotional and contextual cues, not definitive evidence that performance differed among sections. The authors noted that future training data should include professionally worded but competency-inappropriate responses and borderline examples from adjacent performance levels. Faculty feedback was favorable despite some technical friction.

The seven participants included three men and four women with 20 to 30 years of clinical and teaching experience, and one had also contributed to the scenario and rubric, making the evaluation a feasibility pilot rather than fully independent validation. The overall NASA Task Load Index mean was 47.19 with a standard deviation of 15.15, indicating moderate perceived workload. Physical demand was lowest, while effort and frustration were the highest-rated dimensions.

The System Usability Scale mean was 63.75 with a standard deviation of 15.86, slightly below the commonly cited benchmark of 68 for average usability, and individual scores ranged from 45 to 90. Participants completed the interactive scenario in a mean of 19.96 minutes with a standard deviation of 4.45, excluding the tutorial and automated debriefing. No participant withdrew, although a technical failure could require restarting the application from the beginning.

The educational ratings were stronger. On the modified Simulation Effectiveness Tool, the mean was 2.50 with a standard deviation of 0.43 on a three-point scale, indicating high perceived effectiveness. Faculty members gave particularly high ratings for patient communication, clinical decision-making, assessment, prioritization and safety, while medication knowledge and pathophysiology received lower average ratings, consistent with the simulation’s emphasis on communication and de-escalation.

A three-participant subset scored perceived usefulness on a technology-acceptance survey at a mean of 4.20 with a standard deviation of 0.70 on a five-point scale, but that incomplete result was considered descriptive only. Interviews identified the debriefing and its explanations for automated scores as a major strength, while participants recommended clearer controls, a more intuitive performance hierarchy, better indications of when the debrief had ended and more familiarization with VR. The findings support preliminary feasibility and educational relevance, but the study does not establish student learning benefits, reliable scalability, long-term curriculum effects or improved real-world clinical performance.

Privacy also remains unresolved because learner speech was processed through third-party AI services, and the pilot did not evaluate those data risks.

Subject of Research: Pilot evaluation of an AI-driven virtual-reality simulation for automated assessment of nursing communication and verbal de-escalation skills

Article Title: AI-driven virtual reality simulation for nursing de-escalation training: development, automated assessment, and pilot expert evaluation

Article References: Velarde, A. A., Tyerman, J., & Fallavollita, P. (2026). AI-driven virtual reality simulation for nursing de-escalation training: development, automated assessment, and pilot expert evaluation. Frontiers in Virtual Reality, 7, Article 1933489. https://doi.org/10.3389/frvir.2026.1933489

Image Credits: AI Generated

DOI: 10.3389/frvir.2026.1933489

Keywords: virtual reality, nursing education, de-escalation, conversational AI, BERT, automated assessment, speech-to-text, simulation, machine learning, nursing communication, AI-driven, virtual