AI Listens to Music and Reads Brain Blood Flow to Gauge Hidden Consciousness

One of medicine’s most agonizing questions may have just gained a powerful new ally. Each year, thousands of patients survive devastating brain injuries only to drift into a prolonged disorder of consciousness, a state in which they open their eyes, sleep and wake, yet show no clear sign of awareness. Distinguishing a vegetative state from a minimally conscious state is not academic hair-splitting: the two conditions carry radically different prognoses, and misdiagnosis can shape decisions about treatment, rehabilitation, and end-of-life care for families already stretched to their emotional and financial limits. A team of researchers in China has now built an artificial intelligence system that reads the brain’s blood-flow fingerprints while patients listen to music, and it classifies levels of consciousness with an accuracy of nearly 89 percent, outperforming conventional machine-learning methods and single-stream deep-learning models alike.

The stakes are enormous. In China alone, more than 70,000 new cases of prolonged disorders of consciousness arise annually, generating medical expenditures in the tens of billions of RMB and placing crushing burdens on families and the health system. The clinical gold standard, the Coma Recovery Scale-Revised, depends on behavioral responses and intact motor pathways, and its scoring is vulnerable to observer bias. A patient with locked-in features or severe motor impairment may retain flickers of awareness that no bedside examination can elicit. What clinicians need is an objective, physiology-based readout of the brain that does not require the patient to move, speak, or respond on command.

That is precisely what the new study, published in Applied Intelligence, set out to deliver. The researchers collected data at Luqiao Hospital of the Taizhou Enze Medical Center using functional near-infrared spectroscopy, or fNIRS, a non-invasive technique that shines near-infrared light at two wavelengths, 730 and 850 nanometers, through the scalp and measures how much is absorbed. Because oxygenated and deoxygenated hemoglobin absorb light differently, the modified Beer-Lambert law allows researchers to compute continuous changes in oxygenated hemoglobin (HbO), deoxygenated hemoglobin (HbR), and total hemoglobin (HbT) across the cortex. Compared with EEG, MEG, or fMRI, fNIRS is cheap, portable, tolerant of movement and environment, and well suited to the intensive care unit, where scanning a patient inside a giant magnet is rarely an option.

The study enrolled 17 patients in a vegetative state or unresponsive wakefulness syndrome, 22 in a minimally conscious state, and 20 healthy controls, with diagnoses independently assigned by two experienced clinicians using the CRS-R scale. Each participant lay supine wearing a cap of 13 LED emitters and 15 avalanche photodiode detectors, forming 35 measurement channels that sampled at 11 hertz and covered bilateral frontotemporal and prefrontal cortex. Rather than calling each patient’s name, which pilot testing revealed produced inconsistent responses due to dialect differences and naming habits, the team used music. Each session comprised 18 thirty-second musical excerpts, spanning piano pieces, popular songs, and saxophone solos, each followed by a 30-second rest. Music was chosen deliberately: as a non-semantic stimulus it engages auditory, emotional, memory, and attention networks regardless of language or cultural background, and it induces broader cortical activation than pure tones.

Raw optical signals were converted to optical density values, cleaned of motion artifacts using a sliding-window standard-deviation method with spline interpolation, band-pass filtered between 0.01 and 0.2 hertz with a fourth-order Butterworth filter, and transformed into hemoglobin concentrations with baseline correction against the pre-task rest. Before the model ever saw the data, the raw signals already told a striking story. Vegetative-state patients showed flat, regular traces with modest, uniform fluctuations across channels, a pattern consistent with globally suppressed neural activity. Minimally conscious patients exhibited more complex, channel-dependent dynamics, hinting that some regions were regaining responsiveness. Healthy controls displayed rich, varied, and clearly differentiated signals across all 35 channels, matching the heavy metabolic demands of a fully conscious brain.

The heart of the work is a neural architecture the authors call pDOC-STANet, a spatiotemporal attention network designed to squeeze every drop of information from those signals. Each trial contains 330 time points across 35 channels, and the three hemoglobin species are concatenated along the channel dimension, producing a 105-by-330 matrix. The temporal branch processes this matrix through an improved InceptionTime network: each Inception block runs parallel convolutions with kernel sizes of 10, 20, and 40, capturing short-term spikes, mid-range dependencies, and long-term trends simultaneously, while a residual connection guards against vanishing gradients. The authors then embedded a Squeeze-and-Excitation module in every block, which uses global average pooling and two fully connected layers to learn channel-wise attention weights, letting the network amplify the most informative temporal features automatically.

The second branch treats the same data as an image. Using the Gramian Angular Difference Field, a technique that converts a normalized time series into polar coordinates and computes pairwise angular differences between every pair of time points, each trial becomes a stack of 105 channel-wise GADF maps of size 330 by 330, a tensor of more than 11 million values. This encoding preserves global temporal relationships that a simple reshaping would destroy. A modified ResNet18, with its first convolution widened from three RGB channels to 105, digests these images end-to-end. Finally, an attention-based fusion module projects the two feature vectors into a shared latent space, concatenates them, and learns a softmax weighting over the two modalities for each sample, so the classifier can lean on whichever representation is more discriminative for a given patient.

Evaluated with a strictly subject-independent, stratified five-fold cross-validation, in which all 18 trials from any one participant stayed within a single fold to prevent data leakage, pDOC-STANet reached an accuracy of 88.72 percent, with precision of 88.50 percent and recall of 89.62 percent, on 1,062 non-overlapping samples from 59 participants. Traditional methods lagged far behind: linear discriminant analysis managed only 55.64 percent, decision trees 66.17 percent, and logistic regression, the best classical baseline, 77.44 percent. Among deep-learning approaches, InceptionTime alone achieved 84.96 percent and the GADF-ResNet branch alone 83.46 percent, both beaten by the fusion model with a statistically significant margin confirmed by paired t-testing across folds.

Ablation experiments made the architecture’s logic plain. Naively concatenating the two branches without attention actually degraded performance to 82.71 percent, below either branch alone, showing that multimodal fusion is not automatic and that redundant cross-modal noise must be filtered. Restoring attention weighting lifted accuracy to the full 88.72 percent. Replacing the GADF encoding with a direct two-dimensional reshape cost nearly two and a half points, confirming that the angular encoding preserves genuinely useful temporal dependencies. Modality experiments revealed another surprise: total hemoglobin alone, which reflects overall cerebral blood volume, was the strongest single signal at 85.71 percent accuracy, well above HbO or HbR alone, yet the three-way combination remained best, indicating that oxygenation dynamics carry complementary information. Task-state data collected during music also clearly beat resting-state recordings, which reached only 81.20 percent accuracy, evidence that musical stimulation actively elicits discriminative hemodynamic responses tied to consciousness level.

Crucially, the team did not stop at accuracy. Using LIME for class-specific local explanations and SHAP for global feature importance, they showed that the model’s decisions rest on physiologically plausible territory: healthy-control predictions leaned on HbO and HbR features in left frontotemporal and prefrontal channels, minimally conscious predictions concentrated in bilateral prefrontal regions, and vegetative-state predictions drew broadly on HbR and HbT features across both hemispheres. The researchers are candid about the limits: 59 participants from a single center cannot establish clinical generalization, labels ultimately derive from CRS-R scores, and saliency maps are feature-importance tools rather than causal neuroscience. They plan multi-center validation with more than 200 cases and a lightweight version of the model for portable embedded devices. If those efforts succeed, a bedside cap, thirty minutes of music, and an algorithm could soon give voice, in hemodynamic ink, to patients who cannot speak for themselves.

Subject of Research: Automated assessment of prolonged disorders of consciousness using fNIRS and a spatiotemporal attention deep-learning network

Article Title: pDOC-STANet: A spatiotemporal attention network for automatic assessment of prolonged disorders of consciousness using fNIRS

Article References: pDOC-STANet: A spatiotemporal attention network for automatic assessment of prolonged disorders of consciousness using fNIRS. (n.d.). https://doi.org/10.1007/s10489-026-07542-1

Image Credits: AI Generated

DOI: 10.1007/s10489-026-07542-1

Keywords: fNIRS, prolonged disorders of consciousness, deep learning, attention mechanism, InceptionTime, ResNet, Gramian Angular Difference Field, vegetative state, minimally conscious state, consciousness assessment, LIME, SHAP