Surrogate Human Digital Twins Aim to Model Perception and Cognition for Neuroergonomics

Researchers at Saarland University, htw saar, and Purdue University have proposed a new kind of digital human model that goes far beyond simulating bodies and biomechanics. In a perspective article published on September 21, 2026, in Frontiers in Neuroergonomics, Tajbeed A. Chowdhury, Alexander L. Francis, and Daniel J. Strauss introduce the Surrogate Human Digital Twin, or S-HDT, a computational framework designed to approximate not just what a person is physically doing, but what that person is perceiving, attending to, and believing about the world in real time. The work addresses a persistent gap in neuroergonomics, the study of how the brain and body perform in real-world work environments, where existing digital human models excel at representing muscles, joints, and physiology but fall short when it comes to the messy, dynamic business of embodied cognition.

Human Digital Twins have emerged as a powerful paradigm in industry and healthcare, typically built on musculoskeletal models, physiological monitoring, and biomechanical simulation. These systems can predict body movement, muscle fatigue, workload, or injury risk with impressive fidelity. But the authors argue that such models are fundamentally incomplete for modern environments. Contemporary humans operate in complex multisensory settings characterized by multiple competing sensory streams, information overload, sensory conflict, and sensory strain. Think of a factory floor filled with machinery noise, visual clutter, and constant interruptions, or a virtual reality interface that generates conflicting visual and vestibular signals. These conditions place heavy demands on perceptual and attentional resources, and their effects on behavior cannot be captured by modeling physical characteristics alone.

Current neuroergonomic systems, the researchers note, remain largely descriptive rather than predictive. Many existing approaches analyze physiological signals or behavioral markers after the fact to infer cognitive states such as workload or stress. While valuable, these methods are limited in their ability to anticipate future cognitive states or support proactive adaptation. They treat cognition as a snapshot rather than a continuous, dynamic temporal process, and they often fail to generalize across tasks, individuals, and real-world conditions. Machine learning architectures such as Long Short-Term Memory networks and transformers can model temporal dependencies and enable forecasting, while reinforcement learning supports sequential decision-making, but the authors point out that these approaches are fundamentally passive and data-driven, relying on reward engineering and brute-force trial-and-error exploration. Crucially, none of them account for biological attention as it is understood in computational neuroscience, where attention operates as precision weighting, a mechanism that dynamically adjusts how much influence each sensory channel has on the brain’s internal model of the world.

The S-HDT framework is grounded in active inference, a theory from computational neuroscience that treats perception, cognition, and action as parts of a single unified process of probabilistic inference. Rather than passively receiving sensory data, an active inference system maintains a generative model of the world, continuously predicting what sensory input it should expect and updating its beliefs when predictions fail. The S-HDT applies this principle to model a human operator’s latent cognitive states, including cognitive load, attention, fatigue, emotion, and task engagement. These hidden states are inferred from multimodal observations, which may include physiological measures such as heart rate variability and muscular activity, behavioral cues, and environmental context. The model continuously minimizes the discrepancy between predicted and observed sensory data, producing progressively more accurate representations of the operator’s current cognitive state as an evolving probability distribution rather than a fixed label.

At the mathematical heart of the framework lies variational free energy, a quantity that serves as an upper bound on surprise, the unexpectedness of incoming observations. The free energy expression combines two terms: an expectation term that measures how well the surrogate twin’s current beliefs explain the observed data, essentially its prediction error, and an entropy term that reflects uncertainty about those beliefs. High entropy means several explanations remain possible, while low entropy indicates confidence. By minimizing free energy, the S-HDT approximates Bayesian inference over hidden cognitive states without needing to compute intractable posterior distributions exactly. Importantly, the authors emphasize that free-energy minimization should not be understood as minimizing environmental complexity itself. Surprise is relative to the model’s predictions and the operator’s current goals. In a crowded cocktail party scenario, the operator actively samples task-relevant information through attentional allocation, gaze, and head orientation, and the S-HDT maintains a suitable level of uncertainty for adaptive inference by continuously updating its internal belief distribution.

The framework also introduces an organizational distinction between core and mantle cognitive processes. Core processes correspond to evolutionarily conserved mechanisms that operate across most vertebrate species, including predictive modeling of sensory information, attentional allocation, action control, and threat detection. These provide the fundamental computational infrastructure that enables organisms to survive in uncertain environments by rapidly predicting and responding to external events. Mantle processes, by contrast, represent higher-order cognitive functions strongly shaped by learning, culture, and experience, such as goal management, social cognition, language processing, and symbolic reasoning. The authors stress that this distinction is a functional modeling abstraction rather than a strict anatomical or evolutionary dichotomy, intended to organize processes with different temporal and contextual characteristics while retaining their continuous, bidirectional interactions within the unified generative framework.

One of the most ambitious proposed capabilities is predictive cognitive simulation, in which a sufficiently validated implementation could generate plausible future trajectories of human cognitive states rather than merely estimating current ones. This matters because neuroergonomic environments benefit enormously from predicting performance degradation, workload surges, or attentional lapses before they become apparent in observable behavior. The framework performs continuous cognitive state estimation, constructs environmental mappings that relate task contexts and conditions to cognitive states, supports adaptive interfaces and task allocation strategies personalized to individual users, and functions as an intelligent decision-support system recommending interventions that minimize cognitive overload. The generative nature of the model also enables counterfactual reasoning, allowing the system to answer what-if scenarios by changing prior beliefs or sensory inputs within the generative model and propagating their effects forward, evaluating how changes in task difficulty or environmental stressors would affect physiological metrics without exposing human operators to unnecessary risk.

To illustrate the framework’s behavior, the team presented a proof-of-concept simulation comparing two neuroergonomic scenarios: a quiet environment with low external distraction and a cocktail party environment characterized by multiple competing sensory stimuli. The simulation deliberately instantiated only a restricted subset of the broader framework, focusing on auditory attention and perception-action dynamics, and did not model speech intelligibility, linguistic content, or informational masking. The auditory scenes were represented by 64-dimensional binaural spectral features, with spatial information encoded through left-right gain modulation and frequency-dependent spectral shaping. Sensory inputs passed through a hierarchical scale-space predictive processing model that calculated multi-level prediction errors, and the simulated agent interacted with the environment in a closed loop, updating its heading and attentional focus based on prediction error dynamics. Abrupt increases in free energy induced exogenous attentional shifts, simulating reflexive redistribution of cognitive resources under high load. Across all measured quantities, including free energy dynamics, attentional behavior, tracking performance, and cognitive effort, values were consistently larger in the cocktail party scenario than in the quiet environment, suggesting that rich stimulus environments involve greater uncertainty and prediction errors, requiring continual belief updating. The authors are careful to note that this simulation illustrates the behavior of selected framework components rather than providing empirical validation of human cognitive-state prediction.

The researchers are equally candid about the framework’s structural limitations. A fundamental challenge is the identifiability of latent cognitive states, since multiple internal states may produce similar observable physiological or behavioral patterns. Within active inference, the relationship between biosignals such as cardiac, electrodermal, or muscular activity and the latent affective, motivational, and attentional states they are meant to reveal is typically framed through a Markov blanket, a statistical boundary that renders internal states conditionally independent of the environment. But the authors caution that this boundary is defined relative to a chosen model, level of coarse-graining, and timescale, making it observer-dependent. The mapping from peripheral signals to internal states is many-to-one and not invertible, partly because autonomic signals are non-specific and, for affective states in particular, there are no fixed physiological signatures. Interoceptive inference accounts treat emotion not as a hidden variable passively lying behind the blanket but as a construction built from interoceptive signals together with contextual and conceptual priors, so the same peripheral pattern can support different affective interpretations. Physiological variables are also actively regulated through allostasis, acting simultaneously as evidence about and control over internal states, which makes the boundary partly circular. Active inference, the authors conclude, constrains the state identifiability problem rather than dissolving it, and the S-HDT therefore treats latent cognitive states as probabilistic surrogate representations whose uncertainty should be retained explicitly.

Practical challenges also remain in real-time multimodal data fusion, generalization across users with limited calibration data, and ethical deployment. Continuous inference of cognitive states raises issues of transparency, user autonomy, and data privacy, and the authors argue that predictive neuroergonomic systems must be designed with explainability and human-in-the-loop control, including communicating prediction uncertainty to users and requiring human confirmation before consequential adaptive interventions. Despite these hurdles, the S-HDT represents a conceptual step toward a new paradigm in neuroergonomics, one in which cognitive modeling is dynamic, predictive, and personalized rather than static and retrospective. By unifying perception, cognition, and action under a single computational principle and embedding prediction directly into the inference loop, the framework opens new opportunities for investigating neuroergonomic system design, human-centered artificial intelligence, and adaptive human-machine interaction, with future work focused on multimodal human-data validation and application-specific operationalization.

Subject of Research: A computational framework using active inference to model embodied human perception and cognition in digital twins for neuroergonomics

Article Title: Using a surrogate human digital twin to model embodied perception and cognition for neuroergonomics

Article References: Chowdhury, T. A., Francis, A. L., & Strauss, D. J. (2026). Using a surrogate human digital twin to model embodied perception and cognition for neuroergonomics. Frontiers in Neuroergonomics, 7, Article 1939412. https://doi.org/10.3389/fnrgo.2026.1939412

Image Credits: AI Generated

DOI: 10.3389/fnrgo.2026.1939412

Keywords: human digital twin, neuroergonomics, active inference, free energy principle, cognitive modeling, embodied cognition, multimodal sensing, attention, predictive simulation, human-machine interaction, surrogate model, human-centered AI