The research, led by Thomas Seacrist, Elizabeth E. Walshe, David Grethlein, Megan S. Ryerson, Flaura K. Winston and colleagues at the Children’s Hospital of Philadelphia and partner institutions, demonstrates that time-series clustering, an unsupervised machine learning technique, can automatically identify distinct visual search strategies from eye-tracking recordings collected during simulated driving. Instead of requiring human coders to watch hours of video and label where each driver looked, the approach compares entire streams of gaze data directly and groups drivers whose scanning patterns resemble one another.

The significance of this shift is hard to overstate for the field. Traditionally, characterizing a visual search strategy has meant video coding: trained analysts review synchronized footage of the road scene and the driver’s eyes, mark fixations on areas of interest, and translate those marks into summary measures. The process is labor-intensive, subjective at the margins, and effectively caps the size of datasets a lab can analyze. That cap matters because the consequences of insufficient visual search are severe. A driver who fails to scan far enough ahead may miss a pedestrian stepping out from behind an obstructed view, or may enter a blind curve without the anticipatory glances that experienced drivers deploy.

To test their method, the team collected eye-tracking data from 36 young drivers aged 16 to 24 as they completed a virtual driving assessment, a validated simulated driving test used to gauge performance behind the wheel. The cohort is deliberately focused on a high-risk population. Crash rates among newly licensed and teenage drivers remain dramatically elevated compared with older, experienced motorists, and errors in scanning and hazard anticipation figure prominently among the mistakes that precede serious crashes involving young novices.

At the heart of the method is a measure called dynamic localized coordinate aligned warping, abbreviated DCLAW, an extension of the well-known dynamic time warping algorithm. Dynamic time warping, originally developed for aligning spoken word recordings, allows two time series that unfold at different speeds to be compared by stretching and compressing them along the time axis. DCLAW adapts this idea to eye-tracking data, which arrive as rapidly sampled coordinates of gaze position and rarely align neatly between one driver and another. Two drivers might sweep their eyes across the same sequence of road regions but at slightly different moments or paces; a naive comparison would call them dissimilar, while DCLAW can recognize the underlying strategy as essentially the same.

Once pairwise similarities between all drivers’ raw gaze streams had been computed, the researchers applied k-medoids clustering, an unsupervised algorithm that partitions data into groups organized around actual representative examples called medoids. Because the method is unsupervised, it requires no preconceived categories of good or bad scanning behavior. The number and structure of the clusters emerge from the data itself, and the researchers then examined each cluster’s medoid, the driver whose time series sits at the center of the group, to characterize what defined that particular visual search strategy.

The results showed that time-series clustering successfully identified generalizable visual search strategies during a curved roadway scenario, one of the more demanding situations in driving. Curves demand anticipatory glances toward the tangent of the bend and disciplined checking of the lane ahead, and differences in how young drivers allocate attention there are linked to crash risk. The fact that clustering raw data recovered meaningful, generalizable strategy groups in this setting suggests the technique can capture behaviorally important variation without any manual preprocessing of the eye-tracking signal.

The practical implications extend well beyond the driving simulator. Because the method removes the need for labor-intensive manual coding, it opens the door to analyzing far larger and more diverse datasets, the kind of scale needed for findings to generalize across populations, driving environments and research questions. Larger samples could reveal how visual search strategies differ by age, experience, fatigue, distraction or neurological condition, and could support the evaluation of training interventions designed to teach novice drivers to scan more like experts. Similar approaches could prove valuable in aviation, air traffic control, construction safety, medicine and any domain where situational awareness depends on where people look and when.

The study also reflects a broader trend in behavioral science, in which methods developed in the data mining community, including time-series clustering, shapelet-based classification and related techniques, are being redeployed to make sense of rich, high-frequency behavioral recordings. Eye-trackers have become cheaper and more ubiquitous, generating torrents of gaze data that conventional analysis pipelines were never designed to handle. Techniques that operate directly on raw time series, rather than on heavily processed summaries, promise to preserve the temporal structure of behavior that those summaries often discard.

The authors have made their data and code publicly available through the Children’s Hospital of Philadelphia’s GitHub repository, lowering the barrier for other teams to adopt and extend the approach. For a research area long constrained by the slow economics of video coding, the demonstration that raw eye-tracking streams can be clustered into interpretable visual search strategies marks a genuine methodological milestone, one that could accelerate the science of how humans take in the visual world during complex, safety-critical tasks.

Subject of Research: A scalable time-series clustering method for characterizing visual search strategies from raw eye-tracking data in young drivers

Article Title: A scalable method for characterizing visual search strategies: A novel application of time-series clustering to raw eye-tracking data

Article References: Seacrist, T., Walshe, E. E., Grethlein, D., Ryerson, M. S., & Winston, F. K. (2026). A scalable method for characterizing visual search strategies: A novel application of time-series clustering to raw eye-tracking data. Behavior Research Methods, 58(10), Article 290. https://doi.org/10.3758/s13428-026-03154-2

Image Credits: AI Generated

DOI: 10.3758/s13428-026-03154-2

Keywords: eye-tracking, visual search strategy, time-series clustering, machine learning, situational awareness, young drivers, driving simulation, k-medoids, dynamic time warping, hazard perception, Behavior Research Methods, driver safety

Reading may feel effortless, but every line of text triggers a complex series of decisions. The eyes jump between words, pause on difficult passages, skip familiar material and sometimes return to earlier sentences when meaning becomes unclear. Researchers at Aalto University and international partner institutions have now developed an artificial intelligence model that reproduces these decisions with unprecedented accuracy, offering a detailed account of how people allocate attention while reading.

The model is designed not merely to imitate eye movements, but to explain the reasoning behind them. It uses reinforcement learning, an approach widely used in robotics and game-playing systems, to learn how to direct its gaze through words, sentences and paragraphs. Rather than copying patterns from a database of human eye-tracking recordings, the system is trained to decide which piece of text to examine next in order to build the most useful understanding within a limited amount of time.

The research, published in Nature Human Behaviour, is based on a principle known as resource rationality. This theory proposes that the human mind constantly balances the benefits of gathering more information against the cost of spending time and mental energy. During reading, that calculation may determine whether a person carefully studies an unfamiliar word, moves quickly across a predictable sentence or backtracks to resolve a confusing idea.

“Our brain is constantly deciding where to look, what to skip, and when to backtrack—spending attention like a budget to maximize understanding,” says Professor Shengdong Zhao of City University of Hong Kong, one of the researchers involved in the work. The new model represents this process at several interacting levels. It can decide how to handle an individual word, how to navigate a sentence and how to move through the wider structure of a document.

This hierarchy is central to the model’s technical design. At the word level, the system evaluates whether a term is familiar, informative or essential to the meaning of the passage. At the sentence level, it assesses how the current sentence fits with what has already been read. At the text level, it maintains a condensed representation of the document’s overall content. If an important word or clause is missing from that internal representation, the model can direct its simulated gaze back toward the relevant section to recover the information.

The researchers also introduced adjustable reader characteristics into the system. These parameters include language knowledge, memory capacity, visual abilities and eye-movement speed. The result is not a single rigid reader, but a flexible model that can represent different reading profiles. A fast reader with strong memory may move rapidly through a passage, while someone with limited working memory may revisit earlier sentences more frequently to maintain a coherent understanding.

Earlier computational approaches typically learned from large collections that paired text with eye-tracking data. Such systems could reproduce common patterns, including where readers tended to pause, but they often struggled to generalize beyond the language, text type or reading conditions represented in their training data. According to Professor Antti Oulasvirta of Aalto University, those models primarily mimicked behavior. The new approach instead gives the system an objective—understand as much as possible under specific time and cognitive constraints—and allows reinforcement learning to discover effective strategies.

To train the model, the researchers placed it in an environment containing millions of texts. The system received feedback based on how well it retained and represented the meaning of what it read. Over time, it learned to optimize its simulated eye movements, choosing when to move forward, when to slow down and when to return to previously viewed material. When the researchers compared its decisions with real human eye-tracking data, the model closely mirrored the behavior of readers and could be adapted to resemble different kinds of readers.

The achievement could have consequences far beyond laboratory models of cognition. A system that can predict how people allocate attention may help create augmented-reality displays that automatically adjust the pace, layout and density of text. Smart glasses, for example, might present information in a way that suits a user’s reading speed, memory limitations or immediate environment. Digital documents could also be redesigned dynamically, highlighting essential information or restructuring difficult passages without changing their underlying meaning.

The researchers envision applications in education, accessibility and professional communication. A complex legal document could be transformed into versions suited to readers with different levels of language proficiency, while real-time systems could present essential information to drivers without creating dangerous distractions. The team plans to investigate whether the model can support people with dyslexia or low language proficiency. If successful, the technology could mark a shift from mass-produced text toward reading experiences tailored to individual minds, situations and goals.

Subject of Research: Not applicable

Article Title: Hierarchical Resource Rationality Explains Human Reading Behavior

News Publication Date: 10-Aug-2026

Web References: https://doi.org/10.1038/s41562-026-02534-0

References: Nature Human Behaviour, DOI: 10.1038/s41562-026-02534-0

Image Credits: Aalto University

Keywords: artificial intelligence, reading, eye tracking, reinforcement learning, human cognition, resource rationality, augmented reality, personalized text, dyslexia, Aalto University