How the Brain Unscrambles a Visually Warped Alphabet

When you read the word in front of you, your visual system performs a feat of biological engineering so fast and so effortless that it feels like nothing at all. Light patterns hit the retina, and within a fraction of a second your brain has extracted the shapes of individual letters, matched them against stored templates, and delivered meaning to consciousness. But that apparent effortlessness hides a delicate chain of neural projections, each of which must carry spatial information from one stage of the visual pathway to the next with remarkable fidelity. A new computational study published in PLOS Computational Biology by Raffles Xingqi Zhu and Alex S. Baldwin asks a deceptively simple question: what would happen to human vision if those projections were slightly imprecise, so that signals arriving at one cortical stage were subtly scrambled in space before being passed along?

The question matters because the geniculostriate pathway, the main conduit carrying visual information from the eye through the thalamus to the cortex, is built from parallel neuronal projections that each originate at a particular retinal location. If the wiring between anatomical areas were perfectly precise, positional information would be preserved from stage to stage. But biological wiring is never perfect, and any imprecision in these projections would place hard constraints on the visual system’s ability to perform tasks that require knowing where things are, such as identifying the letters of a word. Zhu and Baldwin set out to quantify exactly how much such spatial scrambling would hurt performance, and where along the visual pathway the scrambling would be most damaging.

To do this, the researchers built their analysis around a canonical model of the early visual cortex, focusing on the first orientation-tuned mechanisms in the processing hierarchy, which are analogous to the simple cells found in primary visual cortex, or V1. These cells respond to oriented edges, such as the vertical stroke of a letter T or the diagonal of an N, and they sit at a crucial junction in the visual pipeline. The authors considered two distinct points at which scrambling could occur. The first is the input to these orientation-tuned mechanisms, before orientation is even extracted; the second is the output from those mechanisms, after orientation information has already been computed. They refer to the first case as subcortical scrambling and the second as cortical scrambling, and the distinction turns out to be far from academic.

The methodological heart of the study is a wavelet decomposition and resynthesis algorithm that the researchers developed to mimic these two kinds of distortion. Wavelets are localized mathematical building blocks that can represent an image as a collection of small, oriented patches at different positions and scales, which makes them a natural stand-in for the receptive fields of orientation-tuned neurons. By decomposing a letter image into wavelets and then resynthesizing it after perturbing the spatial positions of those components, the team could generate distorted letters that simulated either subcortical or cortical scrambling. Crucially, the two manipulations produce visibly different results: scrambling the input to orientation mechanisms injects orientation noise into the image, while scrambling the outputs preserves the orientation content but jumbles where each oriented fragment ends up.

With their stimulus-generation machinery in place, Zhu and Baldwin measured how human observers performed on letter identification when the letters had been distorted by the two types of scrambling. Participants viewed the scrambled letters and attempted to identify them, allowing the researchers to derive identification thresholds, the level of distortion at which observers could still perform the task at a criterion level of accuracy. The results showed that subcortical and cortical scrambling have distinguishable effects on both the perceived noisiness of the letters and on the identification thresholds themselves. In other words, the two kinds of distortion are not perceptually interchangeable: the human visual system treats them differently, and the difference is measurable both in how noisy the letters look and in how hard they are to read.

To understand why, the researchers turned to convolutional neural networks, the workhorses of modern computational vision. They assembled a suite of pre-trained and custom-built CNNs and trained them on the same scrambled stimuli that the humans had seen. By comparing the distortion level at which humans could just identify a letter with the level at which a network could do the same, the team computed a relative efficiency, expressed as the ratio of human to CNN thresholds. This efficiency metric is a standard tool in vision science because it separates two sources of performance limits: the information actually available in the stimulus, and the observer’s ability to extract that information. A human who needs twice the distortion tolerance of an optimal machine observer is said to be only half as efficient.

The comparison produced a striking and somewhat counterintuitive pattern. Relative efficiency, calculated against CNNs trained on the scrambled stimuli, was higher for cortical scrambling than for subcortical scrambling. On the surface, that suggests humans cope relatively well when the distortion happens after orientation has been extracted. But when the researchers modeled human inefficiency in a different way, by reducing the proportion of wavelets available to the CNNs and asking how much of the neural representation had to be removed to match human performance, the picture flipped: humans were less efficient under cortical scrambling than under subcortical scrambling. The two analyses, taken together, reveal that the direction of the human advantage depends on which kind of distortion is in play, and that the flip is informative about the underlying neural computation.

The resolution of this apparent paradox, the authors argue, lies in what each type of scrambling does to the information content of the stimulus. Cortical scrambling produces stimuli that are orientation-redundant: the oriented fragments are still present and consistent, just displaced, so the visual system can pool over many redundant orientation measurements to recover the letter’s identity. Subcortical scrambling, by contrast, produces orientation-noisy stimuli, in which the orientation signals themselves are corrupted and cannot be recovered by pooling. The efficiency differences show that humans are better at processing orientation-redundant stimuli than orientation-noisy ones. Human vision, in other words, excels at integrating consistent orientation evidence across space but struggles when that evidence is internally inconsistent, a pattern that no amount of clever pooling can fully repair.

From this pattern, Zhu and Baldwin draw a hypothesis about cortical architecture. The differences in efficiency between the two scrambling conditions, they propose, reflect differences in the integration properties at the input and output stages of simple cells in the cortex. At the input stage, where signals from the thalamus converge onto orientation-tuned neurons, the integration may be relatively tolerant of positional imprecision, because the cell’s receptive field pools over a spatial neighborhood and can average out small displacements. At the output stage, where the orientation signals are passed onward to downstream areas, the integration may operate differently, and positional jumbling of already-extracted orientation features may be harder for the system to undo. The asymmetry in human performance thus becomes a probe into how information is combined at successive stages of the early visual hierarchy.

Beyond its implications for basic neuroscience, the study offers a template for how computational models and human psychophysics can be combined to test hypotheses about neural coding that cannot be directly observed. By constructing stimuli that isolate a specific computational manipulation, measuring human behavior on those stimuli, and benchmarking that behavior against machine observers with known properties, researchers can infer properties of the biological system that would otherwise remain hidden. The work also speaks to a broader question in vision science: the human visual system is not a perfect measuring device, and its inefficiencies are not random. They are structured, task-dependent, and revealing. When letters are warped by physiologically inspired spatial scrambling, the way our performance degrades tells us something fundamental about how the brain wires itself to read the world, one oriented edge at a time.

Subject of Research: The effects of physiologically inspired spatial scrambling at early stages of the human visual pathway on letter identification

Article Title: Identification of letters distorted by physiologically-inspired spatial scrambling

Article References: Zhu, R. X., & Baldwin, A. S. (2026). Identification of letters distorted by physiologically-inspired spatial scrambling. PLOS Computational Biology, 22(10), e1014856. https://doi.org/10.1371/journal.pcbi.1014856

Image Credits: AI Generated

DOI: 10.1371/journal.pcbi.1014856

Keywords: visual neuroscience, spatial scrambling, letter identification, V1 simple cells, wavelet decomposition, convolutional neural networks, relative efficiency, geniculostriate pathway, orientation tuning, PLOS Computational Biology, psychophysics, cortical processing