Conceptual Framework Proposes When AI-Mediated Morality Becomes Extended

A 2026 philosophical perspective argues that the growing integration of aligned large language models into everyday life may do more than help people reason: under specific conditions, AI could become a constitutive part of how humans deliberate, justify choices and form moral habits. The proposed concept of “extended morality” does not grant machines moral agency or final authority. Instead, it describes a sociotechnical process in which systems help structure moral judgment while people remain responsible for setting ends, evaluating principles and accepting or rejecting the norms embedded in those systems. The paper presents a conditional framework rather than reporting a new experiment, and its central claim remains to be tested empirically across professional, educational and private settings.

The proposal draws on 4E cognition, a family of theories that treats cognition as emerging through relationships among embodied organisms, social environments and tools rather than as an isolated process inside an individual mind. Under the framework’s “parity principle,” an external process should not lose its cognitive status merely because it occurs outside the head: if writing down reasons is part of thinking, then a reliably integrated digital tool could likewise participate in cognition. The paper extends this reasoning from cognition to moral deliberation, arguing that weighing reasons, anticipating objections and articulating justifications may acquire similar distributed forms. Unlike a notebook that mainly records a person’s existing commitments, an aligned chatbot can also contribute independently structured normative language, making the moral case more demanding than a simple analogy to familiar artifacts.

Five conditions determine whether human–AI interaction qualifies as extended morality rather than occasional assistance or scaffolding. The system must be reliably available within a person’s routine practice; its normative outputs must usually be accepted as prima facie reasons without case-by-case verification; and its contributions must become functionally integrated with the person’s own justificatory process. The fourth condition is counterfactual impairment: removing the AI would significantly degrade the quality, scope or stability of the relevant moral reasoning. A fifth, specifically normative condition requires the coupling to preserve human authority. If a person abdicates evaluation altogether, the result is not extended morality but moral substitution. Only when all five conditions operate together over time, the paper argues, can the interaction be described as constitutively extended.

Alignment is the technical mechanism through which the proposed feedback loop begins. Supervised fine-tuning, reinforcement learning from human feedback, constitutional rule sets and prompt-level constraints can shape a model’s statistical tendencies toward cautious, respectful, fair or harm-avoiding responses. These orientations are derivative: they trace back to human values, training decisions and institutional priorities rather than to an independently developed moral purpose. Yet they can become stable across contexts, much as repeated action shapes stable dispositions in accounts of moral habituation. The authors stress that an AI system is not virtuous in the experiential or Aristotelian sense. It has no lived experience, affective attunement or practical wisdom, but its procedural constraints can repeatedly place certain responses, arguments and discursive boundaries before users.

That stability is central to the coevolutionary model. Human practices generate communicative data; those data enter training corpora and alignment procedures; models operationalize patterns drawn from that material; users adapt their behavior after encountering the resulting outputs; and their altered behavior produces new data for subsequent updates. Normative expectations can therefore be amplified, filtered or weakened across iterations, even though no participant necessarily intends to change society’s moral vocabulary. The paper cites experimental evidence that advice from large language models can influence moral judgments even when users know the advice came from an algorithm, with people tending to underestimate that influence. Other cited work indicates that algorithmically suggested replies can change the emotional tone of conversations and that co-writing with opinionated language models can alter the views participants subsequently express. These studies support the psychological premise of adaptation, but do not by themselves prove constitutive extension.

The paper separates three functions that are often run together. Normative authority is the capacity to set, endorse and revise moral principles, and it remains irreducibly human. Normative mediation is the way AI structures, filters and guides deliberation by supplying candidate arguments, articulating constraints and modeling acceptable forms of reasoning. Normative stabilization is the gradual formation of durable expectations, justificatory styles and thresholds of acceptability through repeated interaction. The proposed framework therefore allows moral mediation to become sociotechnically distributed without claiming that responsibility has migrated into software. It also distinguishes this process from the idea of distributed morality, which concerns outcomes produced by aggregating many actions: extended morality focuses instead on how the components of deliberation and moral formation are constituted and stabilized over time.

Potential benefits depend on the quality of the feedback system. Systems that explain refusals, acknowledge uncertainty and invite counterargument may provide prompts for reflection, while fluent responses can also narrow the range of reasons people encounter. A particularly important risk is normative homogenization. If AI-mediated discourse increasingly recycles AI-shaped patterns of justification, minority moral vocabularies and argumentative styles may lose room within successive training cycles. The paper relates this possibility to model-collapse research, in which training on increasingly machine-generated material can reduce distributional diversity, although the moral application is an analogy rather than a demonstrated experimental result. Governance must therefore preserve pluralism, including continued human input and protection for dissenting moral languages, while the concentration of large-scale AI development in a small number of corporate organizations raises further questions about democratic legitimacy and geopolitical power.

Three safeguards are proposed. First, asymmetrical clarity requires that users and institutions distinguish fluent moral language from genuine moral authority. Alignment procedures, training objectives and embedded value hierarchies should therefore be transparent. Second, contestability means that people affected by AI-mediated moral environments must be able to question and revise the interpretations encoded in them. Third, reflexive education should cultivate what the paper calls “AI maturity”: the capacity to use AI effectively while recognizing that its values are operational proxies rather than independent moral insight. This maturity concerns when to consult, endorse, contest or disengage from a system. It extends practical wisdom from choosing the right action to governing the tools that structure moral choice, helping preserve autonomy without requiring complete separation from technology.

Even carefully designed systems may promote moral deskilling, a concern raised in the paper. If AI routinely supplies reasons, anticipates objections and resolves dilemmas, people may lose the effortful practice through which judgment is developed. The authors treat this as a contingent outcome of poorly structured coevolution rather than an inevitable law of technology: systems that expose their reasoning, flag uncertainty and welcome challenge could become practice environments, while opaque answer machines could encourage passive acceptance. They also acknowledge three central limitations. The account is conceptual, so the prevalence of the full coupling conditions remains unknown; its present focus is on text-based dialogue rather than embodied, multimodal or autonomous agents; and its emphasis on contestability and pluralism reflects a broadly liberal-democratic framework that may need comparison with other political and cultural systems. The proposed framework therefore offers a testable map of AI-mediated moral change, not evidence that machines have become moral or that routine AI use already produces extended morality.

Subject of Research: A conceptual account of how aligned AI systems may mediate and reshape human moral reasoning and habituation.

Article Title: A coevolutionary account of normative human–AI interaction

Article References: Noller, J. (2026). A coevolutionary account of normative human–AI interaction. Discover Artificial Intelligence, 6(1), Article 1089. https://doi.org/10.1007/s44163-026-02130-1

Image Credits: AI Generated

DOI: 10.1007/s44163-026-02130-1

Keywords: human–AI interaction, extended morality, AI alignment, moral agency, 4E cognition, AI habituation, normative mediation, AI governance, moral deskilling, coevolutionary, account, normative