LLMs Show Distinct Risk Preferences in Adaptive Binary Search Study

Researchers at Ben-Gurion University of the Negev and Shamoon College of Engineering have developed an adaptive binary search procedure to measure the risk preferences of large language models (LLMs) in high-stakes decision-making scenarios. Published in Frontiers in Artificial Intelligence, the study tested eight models from four providers using emergency and disaster management vignettes to determine how these systems weigh lower-risk against higher-risk options when expected values are either explicitly stated or left for the model to infer.

The central finding indicates that six of the eight models exhibited median tipping points near the point of equal expected value under both presentation conditions. However, the two Anthropic models, Claude Sonnet 4.6 and Claude Haiku 4.5, showed higher tipping points when the expected value was left to be inferred, suggesting a systematic deviation from expected-value maximization in specific contexts. This deviation highlights that LLMs do not always treat logically equivalent descriptions of a choice as equivalent, a phenomenon consistent with framing effects observed in human decision-making.

Risk preference describes how a decision-maker chooses between options with uncertain outcomes, characterized relative to expected value, which is the probability-weighted average outcome of an option. In the study’s design, a risk-neutral decision-maker is indifferent between two options of equal expected value, while a risk-averse one prefers the safer option and a risk-seeking one prefers the riskier option. When an LLM informs a risk-related decision, its risk preference can systematically favor safer or riskier options, potentially overriding the option with the higher expected value in ways that may not be apparent from the output itself.

The study utilized an adaptive sequential procedure rather than fixed designs like choice lists, which can only locate preference shifts as finely as their preset values allow. The adaptive method adjusts each query based on earlier responses, concentrating measurement near the model’s tipping point. This approach poses a choice between a lower-risk and a higher-risk option, varying the expected value of the higher-risk option across steps to locate the point where the model’s outputs shift to favor it via binary search.

The experimental vignettes placed the models in the role of incident commander, requiring them to allocate limited rescue resources between two sectors. The lower-risk option offered a 60% chance of saving 40 people, while the higher-risk option offered a 40% chance of saving a variable number of people, N. Both options started with the same expected value of 24 lives saved when N was 60. Models were instructed to allocate seven priority points between the options, with the allocation serving as a measure of preference strength, as the odd total prevented an equal split.

The eight models tested included frontier and efficiency variants from OpenAI, Anthropic, Google, and xAI. Data were collected between February and April 2026 using the OpenRouter API with default inference settings. The procedure involved two phases: bracketing, where N was doubled or halved to find bounds, and binary search, where N was set to the midpoint of the bounds to converge on the tipping point. Each step involved 50 independent queries to account for the stochastic nature of model outputs.

Results showed that for six models, the median tipping point lay between 58 and 62 in both conditions, indicating an expected-value premium between -2 and 2. Claude Sonnet 4.6 had a median tipping point of 74 in the implicit condition and 66 in the explicit condition, with a shift interval of 4 to 8 that excludes zero. Claude Haiku 4.5 had medians of 80 and 62, respectively, but its shift interval included zero, indicating less precision in the estimate. The other six models shifted by four or less in either direction, suggesting their preferences remained stable regardless of how the expected value was presented.

At the query level, deviations from expected-value maximization concentrated in four models. The two Anthropic models made only risk-averse lower-expected-value choices, preferring the lower-risk option even when the higher-risk option had a higher expected value. Conversely, the two Google models, Gemini 3 Flash and Gemini 2.5 Flash Lite, made almost only risk-seeking choices, preferring the higher-risk option when the lower-risk option had the higher expected value. This directional split along provider lines suggests distinct underlying mechanisms or training influences, although the study notes that with only two models per provider, this cannot be definitively attributed to provider-level training choices.

Presenting the expected value explicitly reduced these deviations in all four models over the shared range of values queried in both conditions. The difference in lower-expected-value choice rates between the implicit and explicit conditions was 29.1 percentage points for Claude Sonnet 4.6, 24.5 for Gemini 2.5 Flash Lite, 13.6 for Claude Haiku 4.5, and 12.5 for Gemini 3 Flash. These differences, with bootstrap intervals excluding zero, indicate that the models did not consistently maximize expected value when it was not explicitly stated, implying that the stated value may act as a salient cue or supply a quantity the model did not otherwise compute.

The study has several limitations, including the use of fictional, single-turn, text-only vignettes that limit ecological validity relative to operational decision-support settings. The three scenarios shared a single decision structure and outcome unit, so the measured preferences may not extend to other risk domains or decision structures. Additionally, the success probabilities were fixed at 60% and 40%, meaning the expected-value premium is an index of risk preference under one probability structure and cannot separate the curvature of utility from probability weighting.

The binary search procedure relies on a monotonicity assumption, which was checked and found to hold across all 48 trajectories, as no model preferred the higher-risk option at one value of N and the lower-risk option at a higher value. However, the mean priority-point allocation was locally non-monotonic in 20 of the 48 trajectories, though this did not change the preferred option or search direction. The bootstrap intervals used to estimate uncertainty capture sampling variation among the 50 queries within each step but do not capture variation between independent repetitions of the whole procedure.

This adaptive procedure offers a reusable tool for comparing and screening models for risk-related decision support, applying to any model accessible via a text interface. It localizes the tipping point more finely than a grid of values fixed in advance for the same number of queries, which is crucial for identifying subtle differences in model behavior. Future work could replicate the procedure on open-weight models to test mechanistic explanations and compare it empirically with fixed-grid designs under the same query budget to further validate its resolution and efficiency.

Beyond the core numerical results, the experimental design incorporated specific procedural safeguards to isolate model preferences from training contamination. The three disaster scenarios, covering an earthquake, a wildfire, and a chemical spill, utilized entirely fictional locations, dates, and event descriptions so that outputs reflected the manipulated probabilities and casualty numbers rather than memorized historical incidents. Before deployment, each vignette underwent a two-stage quality review: Meta’s Llama 3.1 model first audited the text for ambiguities and grammatical inconsistencies, followed by manual review by the lead author to correct minor typographical issues without altering the scenario parameters, numerical options, or required response formats. The resulting adaptive search proved highly efficient, with 46 of the 48 experimental trajectories converging in just five to eight steps, representing 250 to 400 total queries, to a boundary span of five or fewer individuals.

A closer examination of the 10,750 individual queries evaluated at values of N other than 60 showed that deviations from expected-value maximization were relatively rare across the broader benchmark, totaling 788 choices, or 7.33 percent. Risk-averse selections accounted for 5.52 percent of all queries, while risk-seeking selections accounted for 1.81 percent. Three models, GPT-5.2, GPT-5 Mini, and Grok-4.1 Fast, made no lower-expected-value choices in either condition, while Grok-4 recorded only a single lower-expected-value decision out of 500 implicit queries. When facing identical expected values at the baseline of 60 saved lives, where expected-value maximization provides no distinct preference, several models displayed systematic directional leans. GPT-5 Mini consistently favored the safer alternative in all six trajectories to finish at 62, Grok-4.1 Fast favored the riskier option across every implicit run to reach 58, and two single trajectories for GPT-5.2 and Grok-4 converged exactly at 60 after generating an even 3.5 mean allocation.

The distribution of priority points also highlighted marked differences in decisiveness and query consistency across model architectures. In 58.8 percent of all valid queries, models selected a 4-3 split, which is the smallest margin the format can express. Claude Haiku 4.5 reached this minimal split in 99.3 percent of its implicit responses, and both Anthropic models produced 4-3 allocations across most queries in both conditions. Consequently, rescaled allocation strength was lowest for the two Anthropic models and Gemini 2.5 Flash Lite, spanning 0.00 to 0.13, and highest for Gemini 3 Flash, Grok-4, and Grok-4.1 Fast, spanning 0.56 to 0.83. Consistency measurements further clarified that Gemini 2.5 Flash Lite’s implicit deviations stemmed from query disagreement, with an average consistency of 0.79, rather than a sustained preference. Although models provided brief written justifications alongside every numerical point allocation, the authors left these explanations unanalyzed to focus strictly on observable decision outputs, depositing the text records in public repositories.

Subject of Research: Measuring risk preferences in large language models using adaptive binary search

Article Title: Measuring risk preferences in large language models using adaptive binary search

Article References: Cohen, I.-C., Littor, N., Elyashar, A., Cohen, O., & Puzis, R. (2026). Measuring risk preferences in large language models using adaptive binary search. Frontiers in Artificial Intelligence, 9, Article 1979431. https://doi.org/10.3389/frai.2026.1979431

Image Credits: AI Generated

DOI: 10.3389/frai.2026.1979431

Keywords: Measuring, risk, preferences, large, language, models, adaptive, binary, search