In a perspective article published in Frontiers in Cognition, Jaap Munneke of Northeastern University, Brian Anthony of MIT, and Jennifer E. Corbett of MIT’s Institute for Medical Engineering and Science and Computer Science and Artificial Intelligence Lab make the case that commercial games are ready-made testing grounds for human perception research. Their argument is not that games should replace laboratory paradigms, which remain indispensable for mechanistic inquiry, but that they can complement them as inherently motivating, sustained environments where genuine cognitive demand arises directly from gameplay structure. Crucially, the researchers say, these environments already exist, and studying them requires only a minimal experimental toolkit of screen recording, eye tracking, and behavioral timing, with no access to source code or developer cooperation required.

The ecological validity problem at the heart of the argument is not merely philosophical. Laboratory tasks reduce the visual world to its essential variables, but real-world scenes are complex, demands are self-paced, and outcomes matter to the observer. Commercial videogames present exactly this kind of environment. A player navigating a fast-paced first-person shooter must continuously parse a cluttered visual scene, suppress irrelevant distractors, and track multiple moving targets. A player managing a large military in a real-time strategy game must maintain and update multiple representations in working memory, plan several moves ahead, and switch flexibly between task demands. These are not approximations of cognitive constructs, the authors argue, but natural instantiations embedded in a context richer and more dynamic than any laboratory task could reasonably replicate.

A common objection to naturalistic research is that unconstrained environments vary too much to allow systematic comparison. Games, the researchers counter, are far richer than laboratory displays but not necessarily less consistent. A given level or map typically presents the same stimuli, the same visual clutter, and the same spatial layout, allowing a level of reproducibility that true naturalistic research often struggles to attain. This consistency and ecological validity stem directly from the affordances, or action possibilities, that games offer users. Elements within games draw the player’s eye to particular locations, reward specific response patterns, and impose sequences of decisions. These affordances emerge from deliberate design choices about what the game demands of its player, and they can be mapped onto cognitive constructs without knowing how the game engine calculates enemy behavior. A researcher does not need engine access to observe that a player of a fast-paced action game must rapidly detect and respond to threats appearing in the visual periphery, a naturalistic example of stimulus-driven exogenous attentional behavior where the affordance structure of the game does the work that an experimental manipulation would otherwise require.

Games have already proven productive for studying perceptual learning, attention, and executive functioning. Foundational work showed that habitual players of action video games outperformed non-players across multiple attentional paradigms, including the attentional blink, multiple object tracking, and the spatial distribution of attention. When non-players were trained on a fast-paced first-person shooter, they showed marked improvements on these tasks, while a group trained on a visually simple puzzle game did not. These findings run contrary to the specificity typically observed in laboratory perceptual learning tasks, suggesting that the affordance structure of the action game was the driving factor. Subsequent work extended the findings to lower-level perception, including visual acuity and contrast sensitivity, and to aspects of spatial cognition such as mental rotation. Successive meta-analyses confirmed that perception and top-down attention are among the cognitive domains most reliably enhanced by action game experience, with effect sizes among the largest in the cognitive training literature.

Executive functioning shows a similar but more component-specific pattern. Working memory, cognitive flexibility, and inhibitory control each respond differently to different games. Training on real-time strategy games has produced improvements in task switching and working memory in older adults and selectively enhanced cognitive flexibility, while training on spatial and shooter titles has produced corresponding gains in spatial and visual working memory. A meta-analysis confirmed that specific design features, rather than broad genre labels, are better predictors of which executive functions a given game will train, and inhibitory control shows a detectable relationship to in-game behavioral metrics like actions per minute. What has been missing, the authors argue, is a systematic account of how to extract and manipulate specific perceptual and cognitive signals as dependent and independent variables over the course of gameplay.

The toolkit the researchers propose addresses this gap by exploiting everything visible and measurable from the outside: the screen, the player, and the built-in options that players use to customize gameplay. Screen recording, an accessible yet underutilized tool borrowed from human-computer interaction research, captures the full behavioral output of the player over time, including every decision, movement, and error. Many gameplay events offer natural choice-reaction opportunities, such as sudden threats or navigation forks, and response latencies extracted through frame-by-frame analysis provide ecologically valid reaction times without game modification or player awareness. With careful post-hoc event coding, screen recordings yield a rich reconstruction of dependent variables like decision timing, response latencies, error patterns, strategy switches, and spatial navigation choices. Corresponding independent variables can be extracted by mapping built-in game features: decision timing varies with the number of choices available, response latencies are affected by the perceptual and attentional demands surrounding a triggering event, and error patterns change as a function of difficulty.

Eye tracking captures a rich behavioral stream from gaze rather than action, with standard metrics extracted automatically via software. Selective attention and processing load can vary as functions of the salience and placement of on-screen elements, reflected in changes in fixation location and duration. Oculomotor efficiency and arousal can be indexed by saccadic velocity, the breadth of attention across the visual field can be inferred from gaze distribution, and both can be modulated by player expertise and scene complexity. Visual search efficiency, in terms of the number of saccades required to locate a target, varies with the density and clutter of the visual scene, while cognitive load derived from pupil diameter varies with difficulty tier and the number of simultaneous demands imposed on the player. Scene parsing and the balance of top-down versus bottom-up control can be indexed by scan path shape, changing depending on whether an active goal or a salient but irrelevant event is present.

Most commercial games also ship with settings that function as ready-made independent variable manipulations. Constructs like spatial navigation strategy might guide a researcher toward open-world or navigation-heavy titles, which can then be evaluated for relevant configurable options like mini-map presence, zoom level, or camera perspective toggle. These settings become specific levers for manipulating a player’s reliance on allocentric, map-based versus egocentric, first-person representations of the environment. As allocentric reliance decreases, route selection should become less direct, backtracking frequency should increase, and gaze should more often shift from the map toward peripheral landmarks, all trackable using screen recording and eye tracking. The authors’ own previous work offers an illustration of this quasi-experimental logic: they found that players’ existing preference for inverting the y-axis on their controllers was linked to individual differences in mental rotation speed and in overcoming stimulus-response compatibility effects. Because these configurable settings can be toggled instantly within the same session, the toolkit enables researchers to transition from traditional between-subjects training designs to high-powered within-subjects repeated measures, directly tracking how an individual’s gaze distribution, response latency, or error patterns shift in real time as affordances are manipulated.

The approach has clear limitations. Without access to a game’s engine, the exact millisecond timing of a player’s response, the precise parameters behind a perceptual manipulation, and the internal state of the game at any given moment may not be directly observable. Even when a configurable option like difficulty tier is manipulated deliberately, its underlying implementation remains opaque and coarser than a classic laboratory parameter. These constraints mean the toolkit is best suited for ecological questions, like how perception operates in a rich and self-motivated natural setting, rather than mechanistic questions about the precise computations underlying a perceptual operation. The two approaches are best understood as complementary, with commercial games testing whether effects established under controlled conditions generalize to richer, self-motivated settings. Realizing this at scale, the authors conclude, requires shared practices such as pre-registering the game, affordances, and hypotheses; standard eye-tracking protocols; agreed conventions for coding recordings; and measuring participants’ prior experience with each specific game. None of this requires new infrastructure. Commercial games do not require redesigning, licensing, or reverse engineering to answer perceptual questions. They require only a screen recorder, an eye tracker, and a researcher willing to state, in advance, which affordance and which configurable options are expected to reveal which perceptual processes.

Subject of Research: Prêt-à-Jouer: commercial videogames as ready-made human perception labs

Article Title: Prêt-à-Jouer: commercial videogames as ready-made human perception labs

Article References: Munneke, J., Anthony, B., & Corbett, J. E. (2026). Prêt-à-Jouer: commercial videogames as ready-made human perception labs. Frontiers in Cognition, 5, Article 1944577. https://doi.org/10.3389/fcogn.2026.1944577

Image Credits: AI Generated

DOI: 10.3389/fcogn.2026.1944577

Keywords: Jouer, commercial, videogames, ready-made, human, perception, labs, scientific research