A Classic Visual Illusion Exposes a Missing Piece in AI Vision

A simple trick of the eye has turned out to be one of the sharpest tests yet of how far artificial intelligence still is from seeing the world the way we do. Researchers at York University have used the motion aftereffect, a classic visual illusion in which a stationary object appears to shift position after an observer stares at moving stimuli, to probe a fundamental difference between primate vision and today’s leading artificial vision systems. Their findings, published in Current Biology, show that a history-dependent computation long thought to be a quirk of biological perception is in fact deeply embedded in the primate brain’s representations of object position, yet is conspicuously absent from the artificial neural networks that currently dominate computer vision.

The illusion at the heart of the study is familiar to anyone who has watched a waterfall. After staring at something moving steadily in one direction for several seconds, a stationary scene viewed immediately afterward can appear to drift, and objects within it can seem slightly displaced in the opposite direction. Nothing in the image itself has changed; the pixels are identical before and after adaptation. What changes is the observer’s brain, which has recalibrated its responses to motion and, crucially, to position. This dissociation between the physical image and the perceptual experience gives scientists an unusually clean window into the computations that the visual system performs, because any shift in what is perceived must originate inside the nervous system rather than in the stimulus.

Kohitij Kar, assistant professor at York University, Canada Research Chair in Visual Neuroscience, and senior author of the study, frames the work as a test of what NeuroAI can deliver when neuroscience and artificial intelligence are brought together deliberately. According to Kar, today’s AI vision systems are impressive but still do not always see the world the way humans do. By designing smart experiments that reveal computations biological vision uses and AI still lacks, he argues, researchers can take those insights and build them into more brain-like artificial systems. That framing matters because the field of computer vision has been built largely around benchmark accuracy: whether a model assigns the right label or the right bounding box. The York team’s results suggest that getting the right answer is not the same as performing the right computation.

The central question the researchers posed was whether artificial neural network models capture the same history-dependent changes in spatial representations seen in biological vision, or whether their internal representations of position reflect only the physical properties of the image. To answer it, first author Elizaveta Yakubovskaya, a graduate student at York, combined recordings from primate visual cortex with human perception experiments. The team used motion adaptation to induce the illusion, making a stationary object appear slightly shifted in position, and then asked whether the brain and the AI models showed the same effect. The comparison required precise measurements of where perceived position and pixel-based position diverge, so that the behavioral relevance of neural position codes could be tested directly rather than inferred.

The results were strikingly asymmetric. Human observers reported the illusion exactly as expected: after adaptation to motion in one direction, the stationary test object appeared displaced in the opposite direction. In parallel, the neural representations of position in the macaque inferior temporal cortex, a high-level visual area critical for object recognition, shifted in the same direction, even though the image presented to the eyes had not changed at all. In other words, the monkey brain encoded the object’s location not in purely retinal or pixel-based coordinates but in coordinates that were perceptually aligned, tracking where the object appeared to be rather than where it physically was. The IT cortex, often studied for what it says about object identity, turned out to be carrying rich, behaviorally meaningful information about object location as well.

When the researchers turned to current artificial vision networks, the picture changed. The models they tested could often determine where an object was located with considerable accuracy, but they did not reproduce the way recent visual experience reshapes that answer. Their position representations remained anchored to the physical properties of the image, unmoved by the adaptation history that so powerfully biased both human perception and primate neural activity. The gap is not a matter of coarse performance metrics; it is a difference in the underlying computation. Biological vision carries a temporal signature, a dependence on what was seen moments before, that these networks simply do not implement when mapping images to spatial estimates.

Why should a systematic perceptual error be treated as a feature rather than a flaw? The answer lies in how neuroscientists think about adaptation. Motion adaptation is one of the most extensively studied phenomena in visual neuroscience, and it reflects the visual system’s active recalibration in response to the statistics of recent input. Neurons tuned to particular directions of motion reduce their responsiveness after prolonged stimulation, and downstream circuits inherit the consequences. That the aftereffect extends to perceived position indicates that these adaptive processes reach all the way into high-level spatial representations, biasing the codes that support judgments about where objects are. A system built to work in a changing world may benefit from such context sensitivity, even at the cost of occasional illusions under artificial laboratory conditions.

The study, titled The macaque IT cortex but not current artificial vision networks encode object position in perceptually aligned coordinates, therefore delivers two contributions at once. Scientifically, it clarifies the role of the inferior temporal cortex in encoding spatial information, showing that its position codes are not static copies of the retinal image but dynamic representations that track perceptual experience. Methodologically, it provides a new benchmark for evaluating dynamic vision models. A model that aspires to human-like vision can now be tested not only on whether it localizes objects correctly on average, but on whether its internal position representations shift in the right way after adaptation, matching both human reports and primate neural data. That is a far more demanding and diagnostic standard than conventional accuracy scores.

For Kar, the work speaks to a growing question in artificial intelligence about whether increasingly capable systems will become more like us or increasingly different from us. If the goal is AI that works with humans and understands the world in more human-compatible ways, he argues, researchers cannot focus only on whether a system gets the right answer; they must also understand the computations that produce human perception and behavior. Neuroscience, on this view, is not merely a source of loose inspiration for architecture design but a discovery engine for the computations themselves, which can then potentially be built into AI. The motion aftereffect, a phenomenon studied for more than a century, has become a concrete example of that pipeline: a perceptual phenomenon, a neural measurement, and an engineering target in a single experimental loop.

The broader implication is that the path to more human-like machine vision may run through the very illusions that reveal the brain’s hidden assumptions. Current networks, trained on enormous datasets to match human labels, have converged on solutions that mimic the outputs of biological vision in many static tasks while diverging sharply in their internal machinery. Illusions like the motion aftereffect expose that divergence in a measurable, reproducible way, because they create a controlled conflict between the image and the percept. As dynamic vision models mature, benchmarks derived from adaptation and other history-dependent phenomena may help close the gap, producing systems whose spatial representations, like those of the macaque IT cortex, are aligned with perception rather than with pixels alone. For now, the humble waterfall illusion stands as evidence that the brain’s quirks encode computational principles that AI has yet to learn.

Subject of Research: History-dependent spatial encoding in macaque inferior temporal cortex and its absence in current artificial vision networks

Article Title: Visual illusion reveals what today’s AI vision is missing – York University study

Article References: Visual illusion reveals what today’s AI vision is missing – York University study. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: visual illusion, motion aftereffect, AI vision, NeuroAI, inferior temporal cortex, macaque, object position, motion adaptation, artificial neural networks, Current Biology, York University, perception