Serotonin Pulls Hidden Levers in the Brain’s Most Connected Structure

Deep inside the brain, wedged between the insular cortex and the striatum, lies a razor-thin sheet of neurons that Francis Crick once proposed might hold the key to consciousness. The claustrum is one of the most densely interconnected structures in the mammalian brain, reaching out to nearly every cortical area and receiving reciprocal input from most of them. Now, a team of neuroscientists at the University at Buffalo has mapped in fine cellular detail how serotonin, the neurotransmitter made famous by antidepressants and psychedelics alike, exerts a surprisingly nuanced grip on this enigmatic hub. Their findings, published in Frontiers in Cellular Neuroscience, reveal that serotonin does not simply turn claustrum neurons up or down. Instead, it plays the structure like an instrument, exciting some cells, silencing others, and quietly turning down the volume on incoming excitatory signals.

The research team, led by Samir Haj-Dahmane, with Renee Rouleau as first author, combined whole-cell patch-clamp recordings, light-based uncaging of serotonin, and precise receptor pharmacology in brain slices from adult rats. This technical arsenal allowed them to measure the electrical behavior of individual claustrum neurons with millivolt precision while systematically blocking or activating specific serotonin receptor subtypes. What they found was a level of cellular specialization that few would have predicted for a structure this small.

The first surprise came from the neurons themselves. Rather than behaving as a uniform population, claustrum neurons split into four electrophysiologically distinct groups. Regular-spiking type I neurons, which made up nearly half of the sampled cells, fire steadily with little fatigue and show a characteristic sag in their membrane voltage when hyperpolarized. Regular-spiking type II neurons fire at similar rates but display a fast afterdepolarization that can trigger bursts of doublets. Strongly adapting neurons, as their name suggests, rapidly dampen their own firing under sustained stimulation. A rare fourth group fires fast and hard, reaching frequencies around 60 hertz. When the researchers filled recorded cells with biocytin and reconstructed their shapes, the regular-spiking type I neurons turned out to have pyramid-like cell bodies with an apical dendrite, while the type II and strongly adapting neurons were triangular with multiple radiating dendrites, all bearing spines.

When serotonin was applied to these cells, the responses diverged along cell-type lines in a striking way. In regular-spiking type I and type II neurons, serotonin could push the membrane potential in either direction, producing depolarization in most recorded cells but hyperpolarization in a subset. In strongly adapting neurons, however, the response was unambiguous: every single one was hyperpolarized by serotonin. Using caged serotonin released by a flash of blue light, the team confirmed that these bidirectional effects could be evoked by brief, physiologically plausible surges of the transmitter, with depolarizations averaging about 4 millivolts and hyperpolarizations close behind at roughly 3.9 millivolts.

The molecular logic behind this push-pull system came into focus through receptor pharmacology. The depolarizing responses vanished when the researchers blocked 5-HT2A receptors with the antagonist MDL 100907, while the hyperpolarizing responses were abolished by WAY 100635, a 5-HT1A receptor antagonist. In other words, the same neurotransmitter carries two opposing messages to the same cells, and the net effect on any given claustrum neuron depends on the balance of receptor subtypes it happens to express. This arrangement mirrors what has been described in the prefrontal cortex, where 5-HT1A and 5-HT2A receptors likewise mediate opposing effects on membrane excitability, but the study is among the first to work out the details in the claustrum.

Serotonin’s influence did not stop at the neuronal membrane. The team next examined the glutamatergic synapses that deliver excitatory input to claustrum neurons, and here they found a second, independent layer of control. Bath application of serotonin cut the amplitude of evoked excitatory postsynaptic currents to roughly half of baseline. Three classic pieces of evidence pointed to a presynaptic site of action: the paired-pulse ratio increased, the coefficient of variation rose, and the frequency of miniature excitatory postsynaptic currents, which reflect single vesicles of glutamate released independently of action potentials, dropped significantly while their amplitudes stayed constant. Together, these results indicate that serotonin reduces the probability of glutamate release from the nerve terminals that feed the claustrum.

Given the claustrum’s famously high expression of 5-HT2A receptors, which are the principal molecular target of serotonergic psychedelics such as LSD and psilocybin, the team suspected these receptors were responsible for the synaptic inhibition. The evidence was decisive. Blocking 5-HT2A receptors with MDL 100907 eliminated the serotonin-induced suppression of synaptic currents, and blocking 5-HT2 receptors more broadly with mesulergine not only abolished the inhibition but unmasked a potentiation, suggesting that other serotonin receptor subtypes exert a weaker, opposing influence. Conversely, TCB-2, a potent and selective 5-HT2A agonist with psychedelic-like properties, reproduced the inhibition on its own, reducing evoked synaptic currents and miniature EPSC frequency in a manner fully reversed by the antagonist.

Perhaps the most intriguing finding concerns where these receptors sit and how they signal. Immunohistochemistry confirmed dense 5-HT2A receptor labeling around claustrum neuron cell bodies and along their processes. Yet when the researchers dialyzed recorded neurons with GDP-beta-S, a reagent that jams G-protein signaling inside the recorded cell, the synaptic inhibition persisted undiminished even though the postsynaptic receptor response was abolished. The canonical downstream pathway of 5-HT2A receptors runs through Gq/11 proteins and phospholipase C, but blocking PLC with U73122 failed to touch the inhibition, and blocking CB1 cannabinoid receptors with AM251, which would have disrupted a retrograde endocannabinoid mechanism, was equally ineffective. The conclusion is that presynaptic 5-HT2A receptors on the glutamate terminals themselves suppress release through a signaling route that departs from the textbook PLC cascade, and the authors acknowledge that identifying that route is a task for future work.

Why does this matter beyond the slice chamber? The claustrum has been implicated in attention, decision-making, multimodal sensory integration, and the coordination of cortical slow-wave activity during sleep. Recent in vivo studies have shown that claustrum neurons projecting to cortex fire most vigorously during the synchronized brain state of non-REM sleep and quiet down during waking and REM sleep, a pattern that mirrors, in inverse fashion, the firing of the dorsal raphe serotonin neurons that innervate the claustrum so densely. The new findings suggest a plausible cellular mechanism: when serotonin tone is low, as it is during deep sleep, the brakes come off both the intrinsic excitability of claustrum output neurons and the strength of their excitatory inputs, potentially allowing the claustrum to drive the synchronized slow waves that sweep the sleeping cortex. When serotonin neurons fire briskly during wakefulness, the same receptors would dampen claustrum activity and gate its excitatory drive.

The psychedelic connection adds another layer of fascination. Psilocybin has been shown to acutely alter the functional connectivity between the claustrum and the brain networks supporting perception, memory, and attention, and psychedelic effects on consciousness are thought to involve the claustrum at least in part. By demonstrating that 5-HT2A receptors simultaneously depolarize major claustrum output neurons and suppress their glutamatergic inputs through a noncanonical presynaptic mechanism, the Buffalo team has provided a cellular blueprint for how a single psychedelic-sensitive receptor could reshape information flow through one of the brain’s great integration centers. As the authors note, the multifaceted and sometimes contradictory effects of 5-HT2A receptors across brain regions likely reflect differences in their subcellular distribution and downstream signaling partners. In the claustrum, at least, serotonin turns out to be less a simple dial than a conductor, coaxing different sections of the neural ensemble into different responses at the same moment.

Subject of Research: Serotonergic modulation of neuronal excitability and glutamatergic synaptic transmission in the rat claustrum

Article Title: Serotonergic regulation of neuronal excitability and glutamatergic synaptic transmission in rat claustrum

Article References: Rouleau, R., Oubraim, S., Cirone, L., Shen, R.-Y., & Haj-Dahmane, S. (2026). Serotonergic regulation of neuronal excitability and glutamatergic synaptic transmission in rat claustrum. Frontiers in Cellular Neuroscience, 20, Article 1951375. https://doi.org/10.3389/fncel.2026.1951375

Image Credits: AI Generated

DOI: 10.3389/fncel.2026.1951375

Keywords: claustrum, serotonin, 5-HT2A receptor, 5-HT1A receptor, patch-clamp, glutamate release, dorsal raphe nucleus, psychedelics, sleep, consciousness, electrophysiology, presynaptic inhibition