The study, published in Frontiers in Ethology by Mahdi Abolhasani and Saman Moghimi-Araghi of Sharif University of Technology in Tehran and Mohammad Salahshour of the Max Planck Institute of Animal Behavior and the University of Konstanz, builds a model in which virtual agents carry two inherited traits. The first is a signal-production distribution: a set of probabilities over one hundred possible signals, each with a fixed production cost drawn once from a uniform distribution. The second is a conditional response rule: for every signal an agent might observe on an opponent, the rule specifies whether the agent cooperates or defects. Crucially, actions depend only on the opponent’s signal. There is no hidden quality variable, no stable cooperative type, and no differential ability to pay. The signals are arbitrary markers whose value, if any, must be created endogenously by the response conventions attached to them.

The researchers tested this machinery across the three canonical two-player social dilemmas: the Prisoner’s Dilemma, the Snowdrift game, and the Stag Hunt. In the Prisoner’s Dilemma, defection strictly dominates cooperation, making it the harshest test. The Snowdrift game belongs to the anti-coordination class, where the best response depends on what the partner does. The Stag Hunt is a coordination game, where mutual cooperation is payoff-favored but risky if the partner defects. Together, these games span qualitatively different strategic worlds, providing a compact test of whether the signaling mechanism depends on a specific payoff structure or carries across contexts.

The simulations, run for twenty thousand generations with populations of roughly sixteen hundred to eighteen hundred agents, produced a striking pattern. In well-mixed populations, the mechanism sustained partial cooperation in the Prisoner’s Dilemma and the Snowdrift game, with asymmetric cooperate-defect interactions accounting for the largest share of outcomes. In the Stag Hunt, cooperation rose quickly toward fixation. No-signal baseline controls sharpened the interpretation: without signals, cooperation collapsed to near zero in the Prisoner’s Dilemma and remained far lower in the Snowdrift game, while the Stag Hunt reached cooperation even without signals because its structure already favors coordination. The signal-response layer, in other words, is doing real work precisely where the dilemma is hardest.

Perhaps the most counterintuitive finding concerns cost. When the researchers plotted signal frequency against production cost, the familiar puzzle reappeared: signals with nonzero costs reached appreciable frequencies. But the corresponding fitness panels revealed that more frequent signals tended to have higher average net payoffs. Signals, the authors conclude, are selected less by their raw production costs than by the cooperative responses they currently elicit. A costly signal persists when the cooperative returns it generates are sufficient to offset its production cost. Cost matters, but not in isolation; its evolutionary effect is mediated by the response convention attached to the signal and by the strategic conflict surrounding that convention.

This mediation produces a dynamic the authors call a signaling war. In the Prisoner’s Dilemma and Snowdrift game, strategic conflict remains strong: signals that attract cooperation also create continuing opportunities for exploitation. No signal-response convention stays secure for long. Selection repeatedly shifts toward newly favorable signals, dominant signal classes rise, become vulnerable as they grow common and exploitable, and are then replaced by other signals that temporarily receive more favorable responses. The result is a moving coevolutionary process in which costly signals help create, stabilize, and then erode temporary cooperative niches. In the Stag Hunt, by contrast, the conflict is weaker. Once a cooperative convention becomes common, mutual cooperation is more attractive than exploiting it, so signals mainly serve to coordinate behavior on an already favorable equilibrium, and signal frequency settles onto a narrower, cheaper, and fitter part of signal space.

Spatial structure strengthened the effect further. When the same games were played on a forty-by-forty square lattice, cooperation rose substantially in the Prisoner’s Dilemma and Snowdrift game, mutual cooperation became more common, and mutual defection was reduced. This is notable because classical work has shown that spatial structure often inhibits rather than promotes cooperation in the Snowdrift game. The opposite pattern here suggests that once signaling is coupled to action, space changes more than encounter frequencies. Local interactions allow signal-response conventions to persist, so agents repeatedly encounter neighbors who attach similar responses to locally common signals. Population structure may promote cooperation not by clustering unconditional cooperators, but by stabilizing local coordination over who is treated as a cooperative partner.

The researchers also tested fluctuating strategic environments, in which the operative game changes probabilistically across encounters. In mixtures of the Prisoner’s Dilemma and Snowdrift game, cooperation changed little as the probability of the harsher game increased, because both component games support similar partially cooperative regimes. In mixtures of the Prisoner’s Dilemma and Stag Hunt, cooperation declined gradually as the Prisoner’s Dilemma became more frequent, with the lattice maintaining appreciably more cooperation deeper into the harsher region. The absence of any sharp jump in either topology shows that the same evolving signal system can move smoothly across changing strategic conditions, shifting between two qualitatively different uses of signaling: convention selection in the Stag Hunt and dilemma mitigation in the Prisoner’s Dilemma.

To understand why the mechanism works in some games but not others, the authors developed two coarse-grained mean-field descriptions. The first, an independence closure that treats signal production and response rules as unlinked at the population level, reproduced the qualitative Snowdrift and Stag Hunt regimes but failed in the Prisoner’s Dilemma, where cooperation declined toward zero. The reason is mathematical: because the temptation payoff exceeds the reward for mutual cooperation and the punishment exceeds the sucker’s payoff, the expected payoff difference is negative for every response profile, so the deterministic dynamics drive all response classes toward defection. A second, rarity-dependent closure added a phenomenological advantage for cooperating against uncommon signals, on the idea that an uncommon signal may initially be associated with more favorable responses before exploitative strategies concentrate around it. This simple correction was sufficient to generate a partially cooperative Prisoner’s Dilemma regime. The authors are careful to note that this demonstrates possibility rather than necessity: linkage disequilibrium, inheritance effects, or spatial assortment could produce similar behavior, and the simulations do not distinguish among these alternatives.

The implications reach well beyond the model. Quality-based signaling accounts, in which costly displays honestly reveal condition, commitment, or trustworthiness, remain plausible for systems where displays track stable individual differences. But this study addresses a different empirical possibility: one in which cues are socially learned, partly arbitrary, or only weakly tied to intrinsic quality, yet become behaviorally important because they organize expectations about whom to help, trust, or avoid. This places the work in direct contact with research on ethnic markers, symbolic labels, group identity, and language-like coordination. It also generates testable predictions: an initially rare marker should receive more cooperation when locally associated with cooperative responses; its spread should be followed by increased exploitation and reduced net payoff; and its decline should coincide with the rise of alternative markers. In spatial populations, clusters of marker use and cooperative response should persist longer, and erosion of a marker’s value should begin near boundaries where exploitative responses are common. Costly signaling, in this view, need not be understood primarily as honest revelation. It can instead be understood as a way of stabilizing a convention long enough for cooperation to become locally advantageous, a mechanism that may underlie everything from fashion cycles in human societies to the shifting badges and displays of animal groups.

Subject of Research: Evolutionary game theory

Article Title: The co-evolution of costly markers and cooperation in social dilemmas

Article References: Abolhasani, M., Moghimi-Araghi, S., & Salahshour, M. (2026). The co-evolution of costly markers and cooperation in social dilemmas. Frontiers in Ethology, 5, Article 1880588. https://doi.org/10.3389/fetho.2026.1880588

Image Credits: AI Generated

DOI: 10.3389/fetho.2026.1880588

Keywords: cooperation, costly signaling, evolutionary game theory, social dilemmas, Prisoner's Dilemma, signaling war, collective behavior, co-evolution, costly, markers, social, dilemmas