Humans Prefer Short Argument-Based Explanations, Landmark AI Study Finds

Artificial intelligence systems are increasingly being asked to do more than make decisions; they are being asked to justify them. One of the most promising routes to transparent machine reasoning is computational argumentation, a branch of explainable AI that models how claims support and attack one another, much as people do in everyday debate. Yet a long-standing question has hovered over the field: when formal definitions built into argumentation systems generate an explanation, do those explanations actually resemble what human beings would produce? A new study published in the journal Cognitive Computation provides the strongest empirical answer yet, and its findings are both reassuring and humbling for the designers of explainable AI.

The research, conducted by Roos Scheffers, Floris Bex, and Matthieu Brinkhuis of Utrecht University, set out to test whether explanation definitions drawn from the computational argumentation literature align with real human explanation behaviour. Computational argumentation represents knowledge as sets of arguments connected by attack relations: one argument can undercut another, and a third argument can defend the first by attacking its attacker. This structure mirrors the argumentative character of human reasoning, in which people naturally seek support for conclusions, weigh objections, and mentally prepare rebuttals in advance of a challenge. Because of this cognitive grounding, the field has long assumed that argumentation-based explanations would feel natural to users. But, as the authors point out, that assumption has been largely untested. Prior empirical work had examined how people evaluate arguments and attack relations, yet only a single earlier study had looked at how people actually explain arguments.

At the heart of the study are three formally defined types of explanation. A sufficient explanation contains the set of arguments needed, together with the topic argument, to guarantee its acceptance against all attackers. A compact explanation is a sufficient one with no redundant members, meaning no proper subset of it would still do the explanatory work. A minimal explanation is the smallest sufficient explanation measured purely by the number of arguments it contains. These categories are nested: every minimal explanation is compact, and every compact explanation is sufficient. Each type embodies a different degree of selectivity, the cognitive-science-inspired principle that good explanations should not overwhelm the recipient with information but should instead zero in on what matters for the conclusion being explained.

To find out which of these definitions best captures human intuition, the researchers recruited 301 participants through the online crowdsourcing platform Prolific, drawing English-fluent adults from 42 countries. The experiment was preregistered, its sample size determined by a power analysis, and it was approved by the Utrecht University Science-Geo Ethics Review Board. Participants worked through eight argumentative scenarios drawn from a pool of twenty that spanned domains including criminal investigations, environmental policy, peer review, and school projects. Each scenario presented a topic argument assumed to be true, one or more counterarguments attacking it, a set of defending arguments that attacked the counterarguments, and one deliberately unrelated argument sharing the same context but playing no role in the dispute.

The task itself was elegant in its simplicity. Participants were asked to explain the conclusion of the topic argument, given the counterarguments, by ticking the boxes of the arguments they believed explained it. They could select as many or as few arguments as they wished, provided they chose at least one. Their selections were then compared against the three formal explanation types and against two statistical baselines: a naive baseline assuming every possible explanation is equally likely to be picked, and a simulated baseline weighted by the observed distribution of explanation lengths among participants. The preregistered hypotheses predicted, in increasing order of selectivity, that people would prefer sufficient, then compact, then minimal explanations more often than chance would suggest.

The results confirmed the hypotheses, though with revealing nuances. Across both argumentation frameworks used in the study, one smaller and one larger, explanations fitting the sufficient, compact, and minimal types were chosen significantly more frequently than the baselines predicted. In the smaller framework, where the minimal explanation consisted of a single argument, participants embraced all three types enthusiastically. In the larger framework, which added a third counterargument and required at least two arguments for a complete defence, the picture shifted: minimal and compact explanations still beat both baselines, but sufficient explanations only outperformed the baseline that accounted for participants’ strong preference for brevity. The take-home message, the authors conclude, is that people prefer short explanations built from arguments directly related to the topic.

That preference for brevity proved remarkably stubborn. In the smaller framework, participants’ explanations averaged just 1.48 arguments, with 62 percent consisting of a single argument. In the larger framework, even though more information was available and a complete defence demanded more arguments, the average grew only to 1.74, an increase of roughly 25 percent. More than half of participants still chose a single-argument explanation. Critically, when they did choose lone arguments, they almost never chose the unrelated one; the distractor argument was picked only about 4 to 7 percent of the time, far below the rate of any relevant argument. This tells the researchers that participants were not simply being lazy. They could tell which arguments mattered and deliberately excluded the ones that did not.

The most striking finding concerned partial explanations. In the larger framework, the majority of participant responses fit none of the three formal types, and most of these consisted of a single defending argument that repelled only one of the topic argument’s several attackers. Participants appeared to identify the argument they judged most important, often the one that fended off multiple attackers or neutralised a unique threat, and stopped there, even though formally complete justification required more. The authors interpret this pattern through the lens of cognitive load theory, the well-established idea that human working memory has limited processing capacity. As the argumentation scenario grew more complex, participants did not scale up their explanations proportionally; instead they simplified, offering shorter and more selective answers rather than absorbing the extra cognitive cost of a full defence. This echoes earlier findings that people adopt simpler reasoning strategies when formal argumentation frameworks become more complicated.

The implications for explainable AI are significant. On one hand, the study validates the field’s foundational intuition: argumentation-based explanation definitions, at least those centred on sufficiency, compactness, and minimality, do capture genuine regularities in human explanatory behaviour, particularly the drive toward relevant, related arguments. On the other hand, the results expose a gap. Formal definitions that require complete, admissible explanations produce output that is systematically longer than what people naturally offer. To close that gap, the authors argue, future work should develop explanation definitions that permit selective, even formally incomplete explanations while preserving as much formal rigour as possible. Such definitions would yield AI explanations that feel less like exhaustive legal briefs and more like the crisp, pointed answers humans actually give.

The study also maps out its own limits. Individual participants varied widely, with about two-thirds changing their explanation length across scenarios while a consistent third always picked a single argument. Responses differed measurably across scenarios, hinting that context and wording shape how people explain, even though no single scenario deviated significantly from the overall pattern in follow-up tests. The experimental setting featured low stakes and no time pressure, and the participants were laypeople rather than domain experts; preferences might shift in high-pressure professional environments such as courtrooms or medical settings, where explanations of AI systems often matter most. The researchers have released their twenty scenarios and both argumentation frameworks as open materials, hoping they will serve as benchmarks for testing new explanation definitions against the explanation behaviour of real human reasoners, a step they see as essential for building AI that is both formally sound and psychologically realistic.

Subject of Research: Empirical testing of human explanation preferences against formal explanation definitions in computational argumentation

Article Title: Empirically Testing Explanation Preferences in Computational Argumentation

Article References: Scheffers, R., Bex, F., & Brinkhuis, M. (2026). Empirically Testing Explanation Preferences in Computational Argumentation. Cognitive Computation, 18(1), Article 109. https://doi.org/10.1007/s12559-026-10654-y

Image Credits: AI Generated

DOI: 10.1007/s12559-026-10654-y

Keywords: computational argumentation, explainable AI, human reasoning, explanation selection, cognitive science, argumentation frameworks, empirical study, cognitive load, selective explanations, artificial intelligence, XAI, formal logic