New Spatial Transcriptomics Tool Maps Gene Expression Across Tissue Boundaries in the Developing Mouse Brain

Tissue boundaries are among the most consequential structures in biology. Where one tissue ends and another begins, cells with sharply different identities must be kept apart, yet they must also communicate, exchange signals, and coordinate their behavior so that organs can grow into their correct shapes. When these borders fail, the consequences range from structural birth defects to invasive cancers. Now, a team of researchers has introduced a computational workflow called REBOSEP that reads gene expression along the seams between tissues, and they have used it to build a detailed atlas of gene activity across five boundary regions of the embryonic mouse brain, published in the open-access journal iScience.

The study, led by Vincent Hammer and Stephan Ossowski, tackles a problem that has grown more urgent as spatial transcriptomics has matured. Modern platforms such as Stereo-seq and Visium HD can, in principle, sequence the entire transcriptome at nearly subcellular resolution, revealing which genes are active at which positions within a tissue slice. But most existing analysis tools focus on identifying tissue domains or layers, treating the boundary itself as an afterthought. REBOSEP, by contrast, is boundary-centric: it takes any existing tissue annotation, measures the physical distance of every data point from an annotated border, and represents each gene’s expression as a continuous signal across that border.

The technical challenge is considerable. Although Stereo-seq spots are minuscule, the number of RNA molecules captured in any single spot is too low for meaningful analysis, so spots are merged into bins, typically of 30 by 30 spots, roughly the size of one to four cells. These bins are formed without regard to where cells actually sit, and bins straddling a tissue edge inevitably contain transcripts from both sides. The result is noisy, sparse data in which clustering algorithms assign random labels to isolated bins and even to empty cavities where RNA has diffused during sample preparation. REBOSEP confronts this by letting users filter out small spurious regions, manually correct annotations, and interactively select the exact stretch of boundary they want to study, then reconstructing long continuous borders even across short gaps.

Once a boundary is defined, the workflow computes the average expression of every gene at each distance interval from the border, producing a table that reads like a cross-section of transcriptional activity. Genes whose profiles are too sparse or inconsistent across replicate samples are filtered out using dynamic time warping, a technique borrowed from time-series analysis that aligns curves by locally stretching or compressing them. A restrictive warping window allows the method to tolerate minor positional shifts of about one bin, roughly 30 micrometers, without conflating genuinely different patterns. Hierarchical clustering then groups genes with similar cross-boundary profiles, and gene ontology enrichment analysis reveals the biological processes each group represents.

To demonstrate the approach, the researchers applied REBOSEP to publicly available Stereo-seq data from E16.5 mouse embryos, analyzing five brain boundaries: dorsal pallium-meninges, midbrain-meninges, lower and upper hindbrain-meninges, and subpallium-meninges. Boundary lengths ranged from about 1,740 to 3,667 micrometers, and expression was traced up to 300 micrometers into the brain and 150 micrometers into the surrounding meninges. The resulting atlas captures the expression pattern of every gene the platform detected, including long non-coding RNAs and poorly characterized Riken genes, many of which now carry a boundary-specific expression signature that offers the first clue to their function.

The deepest analysis focused on the dorsal pallium, the embryonic precursor of the neocortex, and its long border with the meninges, the protective membranes that envelop the brain and play active roles in its development. Clustering revealed striking functional homogeneity. Genes expressed constitutively across all pallium layers were overwhelmingly tied to nervous system and neuron development. Genes active throughout the meninges were enriched for extracellular matrix organization, while genes peaking in the meningeal layers closest to the border showed strong enrichment for angiogenesis, the formation of blood vessels, a process closely linked to the developing blood-brain barrier. On the brain side, a distinct cluster of genes sharply upregulated only very near the border was enriched for synaptic signaling and transmission, and another, with a more gradual falloff into deeper pallium layers, was associated with neuron projection and neuron development.

The researchers then examined how specific signaling pathways are spatially regulated at this border. Wnt signaling, a canonical developmental pathway, showed clear boundary-dependent modulation: the receptors Fzd1 and Fzd2 were inhibited at the border, the Wnt-promoting factor Rspo3 peaked around 75 micrometers, and the cell-cycle regulator Ccnd2, a downstream target, was confined to a narrow band within about 20 micrometers of the interface. Notch signaling, essential for cell fate decisions and neurogenesis, was broadly suppressed across the outer brain layers, with Notch receptors highly expressed in the meninges but reduced toward the border. Cell adhesion molecules mediating neuron-neuron and growth cone-axon interactions were largely confined to a thin band of roughly 50 to 150 micrometers within the brain, with genes such as Cntnap2, L1cam, Ncam1, Nrxn1, Nrxn2, and Ptprd gradually decreasing toward deeper layers.

Robustness checks supported the reliability of these patterns. Repeating the clustering a hundred times with random seeds produced vastly higher similarity within clusters than between them, even when artificial noise was added. Comparing Leiden and Louvain clustering algorithms yielded a Spearman correlation of 0.939 between the resulting pattern distances, and varying the bin size had little effect except at the coarsest resolution of 100 bins, well beyond what is typically used. Crucially, the team validated their findings in an independent Visium HD dataset from E15 to E15.5 embryos, a different technology, different specimens, and a slightly earlier developmental stage. The Visium HD profiles closely reproduced the Stereo-seq patterns, preserving the positions of expression peaks, their relative amplitudes, and their rising or decaying trends.

Perhaps the most provocative finding concerns disease. Many of the genes with strong boundary-specific expression, including Apc, Ccnd2, Tle4, Ctnnd2, L1cam, Nrcam, Cadm1, Cntn1, Nlgn1, and Nrxn2, are known players in cancer biology, autism spectrum disorder, or broader neurodevelopmental syndromes. Tumor malignancy is, in essence, the breakdown of a tissue boundary, and boundary roughness is a recognized marker of invasiveness and poor prognosis. The fact that genes governing adhesion and Wnt signaling are so precisely localized at healthy embryonic borders suggests that tumors may co-opt the same molecular machinery to remodel and invade across tissue frontiers, making these boundary profiles a rich source of candidate mechanisms and therapeutic targets.

The authors are candid about limitations. Gene-level interpretations remain exploratory and hypothesis-generating, the quality of boundary definitions depends on upstream tissue segmentation, current capture rates prevent true single-cell resolution, and all analyzed datasets came from female mice, so sex-specific effects could not be assessed. Still, the resource and the accompanying code, both publicly available on GitHub, lower the barrier for any laboratory to interrogate boundaries in their own spatial transcriptomics data. As spatial omics continues to spread through developmental biology, oncology, and neuroscience, tools like REBOSEP signal a shift in perspective: the most interesting biology may lie not within tissues, but precisely at the lines where they meet.

Subject of Research: Spatial transcriptomics analysis of gene expression patterns across tissue boundaries in the developing mouse brain

Article Title: Spatial transcriptomics profiling across tissue boundaries in the developing mouse brain

Article References: Hammer, V., & Ossowski, S. (2026). Spatial transcriptomics profiling across tissue boundaries in the developing mouse brain. iScience, 29(11), Article 117765. https://doi.org/10.1016/j.isci.2026.117765

Image Credits: AI Generated

DOI: 10.1016/j.isci.2026.117765

Keywords: spatial transcriptomics, tissue boundaries, REBOSEP, developing mouse brain, Stereo-seq, Visium HD, meninges, dorsal pallium, Wnt signaling, Notch signaling, cell adhesion, neurodevelopment