AI Summaries Quietly Turn Pilots’ Hedges Into Hard Facts

When a pilot files a confidential report about a frightening moment in the cockpit, the language is deliberately, carefully uncertain. The crew “may not have seen” another aircraft. A gauge was “possibly” misread. A cause “could not be identified.” These hedges are not stylistic decoration; they are the reporter’s calibration of the evidence, the linguistic machinery that separates what actually happened from what might have happened. Now that artificial intelligence systems are increasingly used to condense these safety reports into short summaries for triage, indexing, and search, a deceptively simple question has become a matter of public safety: do machines keep the hedge? A new study suggests the answer is unsettling on both sides.

Research published in Frontiers in Language Sciences by Simon Baradziej of UiT The Arctic University of Norway tested six open-weight large language models on 42 real incident reports, 30 drawn from the United States Aviation Safety Reporting System and 12 from the United Kingdom Marine Accident Investigation Branch. Each model summarized each report under three conditions: a plain instruction to summarize, an explicit instruction to preserve the reporter’s expressed certainty and uncertainty, and a paraphrase of that preserve instruction. The result was a dataset of 756 generated summaries, of which 650 were scored after empty outputs were excluded. Before a single summary was generated, the hypotheses were fixed: that models would systematically harden uncertainty by default, and that telling them to preserve it would overcorrect into excessive softening.

The default behavior confirmed the first hypothesis with striking consistency. Under the plain prompt, all six models hardened the reporters’ uncertainty, with claim-level hardening rates ranging from 0.59 to 0.85. The mechanism of this distortion was highly specific: explicit hedge words such as “possibly,” “appeared to,” and “apparently” survived plain summarization only about 8 percent of the time, while epistemic modal verbs like “may” and “might” survived far better, at roughly 34 percent. In other words, when a model compresses a report, the adverbial hedges are the first casualties, and their removal is precisely what turns a hedged observation into a flat assertion. One aviation narrative in the study carried ten distinct weakeners, including “probably,” “it seems,” and “assume”; the summary under the plain prompt retained none of them and instead declared an engine failure and a declared emergency as settled fact.

Why would machines behave this way? The study situates the finding within a growing literature showing that language models are poor at expressing uncertainty faithfully. Previous work has found that models largely fail to communicate their own confidence in words, that readers demonstrably over-rely on confidently phrased model outputs, and that human preference data itself is biased against hedged text. Because models are tuned to please readers who dislike doubt, alignment training plausibly pushes them toward assertive phrasing and away from markers of tentativeness. What makes the new result distinctive is that it concerns a different question from model confidence: not whether a model calibrates its own beliefs, but whether it faithfully relays the certainty another writer expressed. A model can be badly calibrated yet faithfully conservative, or the reverse, so the two properties must be measured separately.

Measuring something as subtle as a lost hedge required unusual methodological care. The study used a deterministic lexicon of 143 epistemic markers organized into five categories: hedges, boosters, epistemic modals, evidentials, and approximators, each carrying a direction that raises or lowers commitment. At the passage level, net epistemic commitment was computed as strengtheners minus weakeners per 100 words, so a summary could be scored as hardened or softened relative to its source. At the claim level, the distinct weakeners in each source were aligned against the summary: a dropped hedge counted as hardening, a retained one as preservation, and an added one as softening. The lexicon detector was honestly audited against hand annotation, achieving a precision of 0.63 and a recall of 0.91 overall, with high precision of 0.90 on the lexical hedge and booster categories that carry the main result and low precision of only 0.23 on polysemous modals, where “may” the verb collides with “May” the month.

The measurements were then triangulated against independent references. Two separate language-model judges labeled all 650 summaries, agreeing with each other at a Cohen’s kappa of 0.55, and each independently reproduced the central pattern: dozens of summaries labeled hardened under the plain prompt flipped to softened under the preserve instruction. A human adjudication of 24 source-summary pairs by the author agreed with the automated labels at a kappa of 0.60 and reproduced the pattern on its own, finding five hardened and zero softened among the plain pairs, and zero hardened among all eight preserve pairs. The convergence of a lexical instrument, a marker-alignment measure, two semantic judges, and a human reader is what gives the finding its weight.

Then came the twist that makes the study genuinely surprising. When the models were explicitly instructed to preserve the reporter’s stance exactly, they did not land on faithful preservation; they overshot into softening. Hardening rates fell, but softening became the majority outcome for several models, with retention of source hedges more than doubling to between 0.59 and 0.66. In a cluster-robust logistic regression, the preserve instruction carried an odds ratio of 0.12 for hardening, meaning it cut the odds of presenting tentative claims as definite by nearly ninefold, and this effect held when summary length was statistically controlled. In practical terms, the instruction lowered the predicted probability of hardening from about 0.43 to 0.13, a medium-to-large effect by conventional standards. The obvious remedy, it turns out, introduces a second and opposite error: definite source claims get buried under manufactured doubt.

Crucially, the distortion is not an inevitable by-product of compression. A skeptic might argue that because summaries are shorter than sources, hedges simply get squeezed out like any other word. Three results demolish that account. The preserve-condition summaries were actually slightly longer than the plain ones, yet differed sharply in hardening; the condition effect survived a compression covariate in the regression; and at the category level, hedge deletion under plain summarization was essentially flat with respect to how much the summary was shortened, with a correlation near zero. Hedges are dropped whether the summary is long or short. The loss is a targeted deletion, a disposition of the models rather than an arithmetic consequence of shrinking text, and the same-length summary can be more or less faithful depending entirely on the instruction it receives.

The stakes extend well beyond linguistics. Aviation and maritime safety systems aggregate thousands of incident reports into the trend statistics and causal coding that inform regulatory attention, airworthiness directives, and safety recommendations. A systematic hardening bias in machine summaries would quietly bias which factors appear prominent in that aggregate record, over-representing suspected contributors as established causes, precisely the evidence that drives where regulators and operators direct scarce resources. The opposite error, over-softening, can bury a firm observation under vagueness and hide signals the reporter intended to convey. Both failures are invisible unless each summary is compared to its source, which is rarely done at scale, and a record in which possibilities have silently become findings misleads the very analyses that safety reporting exists to inform.

The study’s author, and the evidence, point toward concrete safeguards rather than despair. Epistemic fidelity should be treated as a first-class quality metric, measured directly with tools like the transparent lexical detector and semantic judges used here, and reported alongside conventional measures of factual consistency. Summaries whose measured certainty shift is large can be routed to human review, keeping oversight burden low while catching the distortions that matter. Deployments should not treat a single preserve instruction as a solution, since the study shows it trades one error for another. More promising approaches include constrained generation that carries across the specific hedges detected in each source, and training objectives that penalize epistemic misrepresentation in both directions. Because the monitoring metrics require no gold labels, they can run continuously on production traffic, turning epistemic fidelity from a one-off research finding into an operational service indicator. The reporters who file these accounts under protection of confidentiality are owed, at minimum, that their maybes survive the machines.

Subject of Research: Uncertainty preservation in large language model summaries of aviation and maritime safety incident reports

Article Title: Do large language models keep the hedge? Uncertainty preservation when large language models summarize aviation and maritime incident reports

Article References: Baradziej, S. (2026). Do large language models keep the hedge? Uncertainty preservation when large language models summarize aviation and maritime incident reports. Frontiers in Language Sciences, 5, Article 1953328. https://doi.org/10.3389/flang.2026.1953328

Image Credits: AI Generated

DOI: 10.3389/flang.2026.1953328

Keywords: large language models, summarization, uncertainty, hedging, aviation safety, maritime safety, epistemic modality, ASRS, MAIB, natural language processing, safety reporting, epistemic fidelity