Study Finds Exploration Drives Human Differences in Uncertain Decision Tasks

Individual differences in how humans navigate uncertain sequential choices are primarily driven by their propensity to explore rather than their sensitivity to rewards, according to a large-scale computational analysis published in iScience. Investigating how people balance the exploitation of familiar options against the discovery of potentially superior alternatives, an interdisciplinary research team discovered that behavioral strategies across a population align along a coherent, low-dimensional manifold. Rather than reflecting distinct learning rates, idiosyncratic reward processing mechanisms, or varied levels of decision noise, the primary axis structuring human choice behavior captures how frequently individuals leave established choices to sample alternative actions. This finding clarifies longstanding debates within cognitive science regarding which latent cognitive processes dominate performance during complex multi-option evaluations.

To investigate normative behavioral variation without imposing restrictive computational assumptions, the investigators examined a cohort of 1,001 participants completing a probabilistic, restless three-armed bandit task comprising 300 sequential trials per individual. In this experimental paradigm, participants repeatedly chose among three distinct options, each governed by an underlying probability of delivering a reward that shifted randomly and unpredictably over time. Because reward schedules fluctuated continuously across trials, successful participants could not simply identify an optimal choice once; instead, they had to continuously infer fluctuating values by integrating recent outcomes while deciding when to exploit the currently favored option and when to explore unchosen targets. Across the entire sample, observed reward probabilities averaged 0.628, substantially exceeding the participant-specific chance baseline of 0.500, confirming that participants systematically tracked dynamic contingencies well above chance levels.

To model individual strategies in a model-free representation, the researchers mapped each participant into a high-dimensional behavioral space based on empirical choice-reward histories. Each individual was defined by a sixteen-element probability vector cataloging all possible two-trial permutations of rewards, reward omissions, stay choices, and switch decisions. However, a major methodological challenge in stochastic decision environments is that differences in experienced reward schedules can inject substantial extrinsic noise into behavioral representations. In this dataset, chance fluctuations in reward probability accounted for 29.1 percent of the total variance across individuals. To isolate genuine cognitive variation from environmental randomness, the authors introduced confound-null principal component analysis, a mathematical technique that orthogonalized the behavioral choice vectors with respect to participant-specific chance reward probabilities before extracting principal axes of variation.

Following orthogonalization, the researchers found that individual differences were concentrated along remarkably few dimensions, with the first three confound-null principal components accounting for 93 percent of the remaining behavioral variance, representing 66 percent of total population-level variance. Visualizing this reduced space revealed a continuous, non-linear cloud of data points curving through the coordinate space in a geometry resembling a chicken wing. Applying multidimensional scaling to this manifold allowed the researchers to linearize the primary axis and rank participants along a single continuous coordinate termed manifold position. Validation on simulated reinforcement learning agents confirmed that this analytical pipeline accurately isolated single generative axes of variation while cleanly filtering out stochastic noise caused by drifting reward walks.

When the authors tested which latent cognitive, computational, or demographic factors predicted a person’s manifold coordinate, exploration emerged as the single most powerful factor. The probability of exploration, inferred through a two-state hidden Markov model classifying trials into exploratory and exploitative states, demonstrated an exceptional linear correlation with manifold position, explaining 77 percent of its variance. In contrast, common model-free metrics such as win-stay decisions accounted for 56 percent of the variance, while model parameters governing learning rates and inverse temperature from standard reinforcement learning models explained less than 6 percent. When forward selection and backward elimination regressions were performed, omitting exploration produced the largest collapse in model explanatory power, decreasing the coefficient of determination by 5.5 percent, whereas removing reinforcement learning learning rates caused changes of less than 0.2 percent.

Because task performance and exploratory choices were inversely correlated across the sample, the researchers performed partial correlation analyses to ensure that manifold position was not merely indexing overall task success or failure. Although individuals occupying higher manifold positions earned lower average reward probabilities—dropping from 0.659 in the lower tercile to 0.584 in the upper tercile—the association between manifold position and exploration remained robust after statistically controlling for earned reward rates. Conversely, when the probability of exploration was regressed out, the relationship between manifold position and earned reward rates ceased to be statistically significant, demonstrating that exploratory drive, rather than task efficiency or reward accumulation, serves as the true organizational axis of human sequential behavior.

Several cognitive and behavioral metrics exhibited pronounced non-linear, inverted U-shaped relationships with manifold position. Both empirical reward sensitivity and estimated reinforcement learning learning rates peaked among participants occupying intermediate positions on the manifold and fell toward both extremes. The authors noted that participants at opposite ends of the behavioral manifold displayed distinct phenotypes: one extreme engaged in near-exclusive exploitation characterized by rigid win-stay choices, while the opposite extreme exhibited near-constant exploration with high switching rates. In both extreme regimes, choices are largely uncoupled from local reward feedback, leading to depressed empirical reward sensitivity and attenuated learning-rate estimates. These non-linearities indicate that reward-learning dynamics operate within broader behavioral modes dictated by the balance between exploration and exploitation.

The study also analyzed the informational structure of participant decisions by calculating the Shannon entropy of choice-reward state distributions and the algorithmic complexity of choice sequences via file compression ratios. Participants positioned higher on the exploration manifold exhibited higher entropy in their choice-reward distributions and generated sequences with reduced compressibility, reflecting greater behavioral unpredictability. However, because these relationships displayed notable non-linearities compared to the strictly linear association observed with hidden Markov model exploration states, the authors concluded that structural unpredictability and randomness are secondary properties of exploratory behavior rather than the primary driver of inter-individual differences.

Evaluating demographic factors revealed that participant age was significantly associated with manifold position and directly governed exploratory dynamics. Although task performance over chance remained stable across age cohorts, exploration declined markedly as age increased. When comparing mathematical models describing this decline, an exponential decay model accounted for age-related shifts in both overall exploration probability and stationary exploration far better than linear or quadratic alternatives, yielding substantially superior Bayesian Information Criterion values. Analysis of transition dynamics within the hidden Markov model demonstrated that the energetic barrier separating explore and exploit states flattened with advancing age, and the stability of exploratory states decayed exponentially, indicating that adults approaching middle age abandon exploratory phases more rapidly than younger adults.

Participant gender exhibited a distinct non-linear relationship with decision-making organization, showing no monotonic correlation with manifold position but predicting dispersion toward the manifold extremes. Self-identified male participants were disproportionately represented at both outer boundaries of the behavioral distribution, whereas female participants clustered more tightly toward the manifold center. Calculations of multivariate centrality within the complete confound-null space confirmed that female participants exhibited greater behavioral typicality than males. The researchers noted that these observations align with the male variability hypothesis observed in cognitive and animal studies, while running counter to hypotheses proposing that female behavioral variability is heightened by endocrine fluctuations.

Methodological control analyses verified that the discovered behavioral manifold was not an artifact of specific feature representations or dimensionality reduction procedures. When the analytical pipeline was repeated using a three-trial choice-reward history window encompassing 64 discrete states, the principal components preserved the same geometric structure and predictor rankings. Splitting trials based on environmental reward richness and evaluating standard principal component analysis without orthogonalization similarly reproduced the core manifold geometry. These checks confirmed that the dominant exploratory axis reflects a fundamental property of cognitive organization in the restless bandit task rather than an artifact of analytical parameterization.

The authors emphasized several key limitations governing the scope of their findings. The sample was cross-sectional with an uneven age distribution, precluding longitudinal causal tracking of cognitive trajectories across the lifespan. In addition, the sample contained limited representation of non-binary and transgender participants, leaving questions regarding broader gender diversity incompletely resolved. Furthermore, because observations were derived from a single probabilistic restless three-armed bandit task, future investigations must establish whether similar low-dimensional manifolds govern behavior in alternative reinforcement learning environments, animal models, or psychiatric clinical cohorts known to exhibit altered exploratory strategies.

Subject of Research: Latent cognitive dimensions and age-related dynamics in sequential decision-making

Article Title: Individual differences in sequential decision-making

Article References: Abbaszadeh, M., Ozanick, E., Magen, N., Darrow, D., Yan, X., Grissom, N., Herman, A. B., & Ebitz, R. B. (2026). Individual differences in sequential decision-making. iScience, 29(12), Article 118213. https://doi.org/10.1016/j.isci.2026.118213

Image Credits: AI Generated

DOI: 10.1016/j.isci.2026.118213

Keywords: sequential decision-making, three-armed bandit, explore-exploit tradeoff, confound-null PCA, multidimensional scaling, hidden Markov model, cognitive aging, reward learning, decision noise, reinforcement learning, win-stay lose-shift, behavioral manifold, activation energy, cognitive flexibility, rescore-wagner model, demographic variation, individual differences