Virtual Reality Steps Into the Assessment Room as AI Learns to Score Communication Skills

Imagine walking into a boardroom, shaking hands with virtual colleagues, navigating a tense project meeting, and delivering a high-stakes presentation to executives — all while invisible sensors record where your eyes travel, how your voice trembles, and which words you choose. That is exactly what 96 Spanish professionals experienced in a new feasibility study that could reshape how organizations evaluate one of the most elusive qualities in working life: communication skill. The research, published in Frontiers in Virtual Reality, built an immersive system called Impactful Communication that drops participants into standardized workplace scenarios and quietly harvests behavioral data to generate automated, machine-scored estimates of seven communication-related competences.

The study, led by Elena Parra Vargas and Mariano Alcañiz Raya of the Institute for Research and Innovation in Bioengineering at the Polytechnic University of Valencia, addresses a persistent problem in personnel assessment. Traditional tools — self-report questionnaires, structured interviews, and human-rated role plays — suffer from weak ecological validity, vulnerability to social desirability bias, and heavy demands on expert time. Virtual reality promises a middle path: researchers can standardize the context and stimuli while preserving enough social realism that genuine communication behavior emerges. When a headset wearer truly feels present with a virtual colleague, the argument goes, their gaze patterns, vocal prosody, and word choices reflect real interpersonal tendencies rather than rehearsed answers.

The technical architecture follows an approach known as stealth assessment, embedded within evidence-centered design. Instead of asking participants to describe their abilities, the system elicits behavior through three connected scenarios delivered on a Meta Quest Pro headset with integrated infrared eye tracking: a scripted project meeting with four virtual team members of differing interpersonal styles, a private one-to-one debriefing with a colleague voicing concerns about team tension, and a formal executive presentation staged in a 360-degree panoramic boardroom. Across roughly 20 to 25 minutes, participants played the role of a team leader coordinating a product launch, and the environment continuously logged gaze data, speech audio, paraverbal descriptors, automatically transcribed speech, and event logs — all synchronized through a common timestamping scheme.

Each of the seven targeted competences — verbal communication, non-verbal communication, paraverbal communication, empathy, assertiveness, active listening, and emotional self-regulation — was mapped to a skill-specific set of features. Verbal communication drew on transcript-derived indicators such as first-person pronoun counts, socio-affective vocabulary, type–token ratios, and mean utterance length. Non-verbal communication relied on area-of-interest gaze metrics: dwell time on each speaking virtual character, gaze switching between team members, and audience scanning patterns during the presentation. Paraverbal features captured pitch, amplitude, RMS energy, speech rate, and pausing behavior frame by frame. Empathy indicators combined accuracy in answering questions about virtual characters’ internal emotional states with preferential gaze toward distressed characters and vocal softening during empathic exchanges.

To generate criterion labels, the researchers triangulated multiple sources. Validated psychometric instruments — the TECA empathy test, the Rathus Assertiveness Schedule, and the Emotion Regulation Questionnaire with its cognitive reappraisal and expressive suppression subscales — provided self-report anchors for the socioemotional constructs. Behaviorally anchored rubrics applied to video recordings of participants’ VR performances supplied labels for the observable communication skills. Each competence was then discretized into Low, Medium, and High categories and modeled as a three-class classification problem, with two supervised algorithms competing: an XGBoost multiclass classifier and a Gaussian Process Classifier with a radial-basis-function kernel. Training used repeated 80/20 train–test splits across 48 random seeds, with SMOTE resampling applied strictly within training data to avoid leakage.

The results were intriguing but deliberately hedged. For verbal communication, XGBoost reached a mean accuracy of about 0.64 against an empirical majority-class baseline of 0.37. Gaussian Process models performed best for non-verbal communication, paraverbal communication, empathy, and active listening, all landing above their baselines. The headline-grabbing numbers came for assertiveness (XGBoost, 0.86 against a 0.55 baseline) and one emotional self-regulation operationalization (0.86 against 0.53). Yet the authors are unequivocal that these figures demand caution: the assertiveness label distribution contained only four participants in the Medium category, and the reappraisal-based self-regulation outcome had a Medium class of just seven. Under such imbalance, high overall accuracy can mask complete failure to discriminate minority classes — a problem the study cannot resolve because fold-wise confusion matrices and class-sensitive metrics were not retained.

This self-critical framing is, arguably, the study’s most valuable contribution. The team explicitly refuses to describe the models as operational scoring engines, noting that questionnaire-derived labels capture self-reported dispositional tendencies rather than enacted behavior, and that the independent rater-level data needed to compute inter-rater reliability for the expert- and peer-rated outcomes were not preserved. They also reject the commonly used balanced 0.33 chance benchmark in favor of empirical majority-class baselines, a more honest standard under skewed label distributions. The paper reads less like a product announcement and more like a disciplined proof of concept: multimodal behavioral traces captured in VR demonstrably contain signal, but the chain of inference from construct to scenario to behavior to score remains only partially validated.

The implications extend well beyond measurement theory. Employment-related AI systems are classified as high-risk under the European Union’s Artificial Intelligence Act, and the authors argue that any future deployment would require model cards, subgroup fairness audits, adverse-impact monitoring, version control, and genuine human oversight. No demographic subgroup analyses were conducted here, so the study offers no evidence about whether the models perform equitably across age or gender groups. The researchers’ summary position is memorable: the algorithm may earn a seat at the assessment table, but not the chairmanship. VR-derived scores, they suggest, should function as decision support within multi-method batteries — enriching validated psychometric scales and human judgment rather than replacing them.

What makes the work compelling as a template is its modularity. The finding that different competences favored different classifiers argues against a single universal scoring model and instead supports construct-specific feature sets, validation evidence, and model-selection criteria within a shared framework. Future iterations will need larger and more culturally diverse samples, retained rater-by-participant matrices for reliability estimation, continuous or ordinal scoring alternatives to brittle Low/Medium/High cutoffs, class-sensitive performance reporting such as balanced accuracy and macro-F1, formal speech-recognition quality metrics, and external validation on independent test sets. Longer-term studies should also test whether VR-inferred competence levels predict real-world job performance, team outcomes, and leadership effectiveness.

Even at this early stage, the vision is striking. Assessment centers, video interviews, and self-report batteries have long struggled to capture communication as it actually unfolds — in the pause before a difficult answer, the glance that acknowledges a struggling teammate, the vocal steadiness held under evaluative pressure. Immersive simulations can, for the first time, standardize those pressure points while measuring the responses at millisecond resolution. The Valencia team has not yet built a validated assessment instrument, and they say so plainly. What they have built is the pipeline: theoretically grounded scenarios, synchronized multimodal sensing, and exploratory classifiers that beat their baselines often enough to justify the next, harder phase of validation. If that phase succeeds, the annual performance review may one day begin not with a questionnaire, but with a headset.

Subject of Research: Feasibility of automated multimodal assessment of professional communication skills in immersive virtual reality

Article Title: Towards automated assessment of communication skills in virtual reality: an exploratory feasibility study

Article References: Towards automated assessment of communication skills in virtual reality: an exploratory feasibility study. (n.d.). https://doi.org/10.3389/frvir.2026.1872244

Image Credits: AI Generated

DOI: 10.3389/frvir.2026.1872244

Keywords: virtual reality, automated scoring, communication skills, eye tracking, stealth assessment, evidence-centered design, machine learning, speech prosody, multimodal analytics, personnel assessment, XGBoost, class imbalance