How AI systems reshape human judgement in mediated society

The idea that artificial intelligence systems merely assist human decision-makers is, according to a new study, one of the most consequential fictions of our time. A paper published in the open-access journal AI & Society argues that AI is quietly restructuring the very conditions under which human judgement is exercised, producing what the authors call “decision substitution” — a phenomenon in which humans remain formally in charge while the practical authority over decisions migrates towards algorithmic systems.

The study, led by Ayodeji Olusegun Ibitoye of the University of Greenwich, together with colleagues at the University of Ibadan and the University of Medical Sciences in Nigeria, does not offer new experimental data. Instead, it builds a theoretically integrated account drawing on philosophy of technology, science and technology studies (STS), human–computer interaction research, legal scholarship and data science literature. Its central claim is provocative: the ethical problem of AI is not primarily bias, transparency or accuracy, but the redistribution of judgement itself — a transformation that existing governance frameworks are structurally unequipped to detect or address.

The authors begin from a widely observed discrepancy. Public discourse and most policy frameworks describe AI as “decision support” — technologies that supply recommendations, predictions or classifications to human decision-makers who retain final say. But empirical research tells a different story. Studies of clinical decision-support systems, welfare risk-assessment tools, algorithmic management platforms and AI-driven educational software consistently show that professionals treat algorithmic outputs as defaults around which decisions are organised, rather than as optional inputs. When deviation from a system’s recommendation requires extra documentation, managerial approval or exposes a professional to blame, deference becomes the rational course of action — even among experts who are technically capable of independent reasoning. This is the well-documented phenomenon of automation bias, but the authors argue it is not merely a cognitive error. It reflects rational adaptation to institutional environments in which algorithmic outputs carry cultural authority and alignment with them is rewarded.

To explain this, the study introduces the concept of the “decision field” — the socio-technical environment within which authority, participation, validation and legitimacy are organised. Traditional accounts of decision-making assume a stable distribution of roles: humans identify information, interpret its significance, weigh competing considerations and assume responsibility for outcomes. AI-mediated environments dissolve that stability. The authors identify three simultaneous reorganisations. Structurally, human actors shift from being judges to being validators: instead of originating judgements through independent deliberation, they review, confirm or approve recommendations generated elsewhere. Functionally, AI restructures visibility — determining which signals are highlighted, which risks are prioritised and which possibilities remain invisible — while also compressing decision timing and shifting the burden of justification. Normatively, legitimacy migrates: decisions become “reasonable” because they align with system outputs, and disagreement requires explanation.

The paper distinguishes three configurations of human–AI decision relations. In decision support, algorithmic outputs are one informational resource among many, and human actors remain the primary authors of decisions. In decision delegation, oversight continues but portions of the process are routinely entrusted to AI, and humans increasingly function as validators rather than originators of judgement. In decision substitution, algorithmic outputs become the primary reference point through which decisions are understood, justified and enacted — even though humans remain visibly present in the process. Crucially, the authors stress that the same AI system can occupy different positions in different institutional contexts; the typology describes arrangements of authority, not technical characteristics of the technology.

Perhaps the study’s most useful contribution is a set of four “threshold conditions” that mark the transition from support to substitution. The first is the institutional cost of disagreement: when dissent against algorithmic recommendations attracts scrutiny, documentation burdens or professional risk, the capacity to disagree survives in principle but withers in practice. The second is the default validity of algorithmic outputs: humans stop asking whether the system’s recommendation is correct and instead must explain why it should not be followed. The third is the reduction of meaningful corrective capacity: oversight exists on paper, but workflows, opacity, time pressure and complexity make intervention largely symbolic — what the authors call oversight that is “procedural rather than deliberative.” The fourth is the separation of responsibility from authority: humans remain accountable for outcomes shaped by systems they neither designed nor control. Individually, none of these conditions establishes substitution; together they describe a socio-technical condition in which human authorship of decisions has been hollowed out.

The paper illustrates this with two vignettes. In a hospital emergency department adopting AI-assisted triage, the system’s risk score reorganises clinical attention, accelerates assessment and redefines what counts as defensible practice. A physician who follows the algorithm and encounters an adverse outcome is seen as having acted reasonably; a physician who overrides it on the basis of contextual knowledge unavailable to the model may face greater scrutiny. In recruitment screening, an AI shortlisting platform begins as an optional input but, over time, rankings acquire presumptive validity, override becomes burdensome, and if a candidate alleges unfair treatment, human recruiters — not the system’s designers — bear the accountability. In both cases, no single moment marks the transition from support to substitution; judgement is displaced cumulatively, through routinised use and institutional incentive.

The consequences the authors describe extend beyond individual decisions. They identify “responsibility asymmetries” as a structural feature of AI-mediated governance: accountability remains attached to frontline actors, often those with the least capacity to influence the systems shaping outcomes, while authority migrates upstream towards algorithmic infrastructures. Governance frameworks built on transparency, explainability or human-in-the-loop mandates, the authors argue, risk creating an “illusion of control” — the presence of a human reviewer does not guarantee meaningful oversight if that reviewer lacks the time, epistemic access, corrective authority and institutional protection needed to contest outputs. In high-pressure settings, human review can degrade into a ceremonial rubber-stamp, formally assigned but practically hollowed out.

The paper also pinpoints where this dynamic is most dangerous: contexts of “indeterminacy.” AI systems operate only on what can be formalised — measured, classified or learned from historical data. Moral ambiguity, relational nuance, tacit knowledge and conflicting values resist formalisation. When decision substitution occurs in precisely those domains, algorithmic optimisation can crowd out the forms of judgement that sustain social meaning, democratic deliberation and care. The authors cite well-documented harms that follow this pattern: risk-assessment tools that flag Black defendants at disproportionate rates, welfare fraud systems that profile the poor, hiring algorithms that screen out marginalised applicants, and clinical algorithms that deprioritise Black patients by encoding historic inequities. Marginalised groups, the paper notes, tend to face more intensive algorithmic scrutiny while having fewer opportunities to contest or override outputs — making substitution not just an ethical issue but a justice issue.

The study’s remedy is a conceptual reorientation it calls “society shaping the algorithm.” Rather than treating human-in-the-loop as a technical safeguard — a person placed at a checkpoint — the authors argue it must be reconceived as a collective and institutional condition. That means, concretely, giving reviewers the time and resources to evaluate outputs; ensuring epistemic access proportionate to their responsibilities; granting genuine corrective authority that can change outcomes; protecting justified dissent from institutional penalty; building mechanisms for collective learning from overrides and errors; and aligning authority with accountability so that actors do not bear full responsibility for decisions they cannot meaningfully influence. Institutions, the authors add, should be required to explicitly declare whether an AI system is intended to support, delegate or substitute for human judgement — a form of transparency that would enable genuine public scrutiny and identify domains in which substitution is ethically inappropriate.

The study concludes with a reframing of the entire ethical debate around AI. The deepest risk, it argues, lies not in machines replacing humans but in the gradual transformation of the conditions under which human judgement is exercised. Some dimensions of judgement — navigating ambiguity, balancing competing values, responding to contexts that resist formal representation — are not failures of computation but expressions of what it means to be human. As AI becomes embedded in the infrastructures of everyday life, preserving human agency will depend less on symbolic oversight and more on sustaining the institutional, cultural and normative conditions that enable deliberation, contestation and responsibility. Reclaiming those conditions, the authors write, is essential if AI is to contribute to human flourishing rather than quietly erode the foundations of ethical and democratic life.