Generative Artificial Intelligence Transforms Coordinated Online Manipulation Across Digital Platforms

A sociological review of research published during the first three years of publicly accessible generative artificial intelligence reveals that synthetic tools are reshaping coordinated online manipulation across elections, political corruption, public health communication, and geopolitical conflicts. Synthesizing empirical findings from the period following ChatGPT’s release in late 2022, University of Naples Federico II sociologist Monica Murero identifies three interconnected developments altering digital influence operations. Generative artificial intelligence has transformed communicative production through scalable, personalized, and multimodal content creation; enabled artificial intelligence-assisted coordination of distributed campaigns; and introduced new communicative vulnerabilities linked to persuasive, human-like synthetic communication. These transformations indicate that coordinated influence is growing more adaptive, persistent, and difficult to identify within evolving human-machine communication networks. By lowering technical barriers to producing convincing messaging at massive scale, generative systems allow manipulative actors to generate tailored text, audio, and visual materials rapidly, fundamentally altering how deceptive online campaigns operate across major digital platforms.

The study analyzes these empirical shifts through the theoretical lens of coordinated inauthentic behavior, contrasting earlier operational definitions with a communication process framework. In 2018, Facebook defined coordinated inauthentic behavior primarily for platform moderation, describing it as coordinated activity where accounts or pages mislead users about their identities or actions. Murero’s framework, initially developed from analyses of coordinated anti-vaccine communication during the COVID-19 pandemic, reoriented the concept from isolated deceptive posts toward adversarial networks using mixes of authentic, fake, and duplicated accounts across multiple platforms. In the generative artificial intelligence era, coordinated inauthentic behavior refers to manipulative tactics where human, automated, or hybrid activity is strategically coordinated to influence public discourse, with inauthenticity residing in the deceptive representation of how participation is organized. Under this definition, neither automation nor coordination alone establishes inauthenticity; authentic accounts participate in manipulation when their centrally coordinated activity is deceptively portrayed as independent, spontaneous civic expression.

Examining communicative production, the review details how generative tools facilitate the rapid and inexpensive generation of large volumes of persuasive, misleading content. The author highlights empirical demonstrations, including research by Menz and colleagues, where publicly available large language models rapidly generated 102 blog articles containing tailored health disinformation. Subsequent studies documented how generative models diversify and adapt persuasive texts to target specific audiences. While early experiments by Hackenburg and Margetts questioned the persuasive edge of artificial intelligence-driven microtargeting, later investigations by Hackenburg and colleagues provided stronger empirical evidence of the persuasive power of generative systems. In addition, the review notes findings from Lucas and colleagues and Wack and colleagues showing that synthetic deceptive content can remain searchable, indexable, and reusable long after an initial campaign concludes, effectively embedding manipulative materials into broader digital information ecosystems where they continue influencing users.

The second major development focuses on how campaign coordination is shifting from manual human operation toward artificial intelligence-assisted automation. Incorporating Hancock and colleagues’ concept of artificial intelligence-mediated communication—defined as computational assistance modifying, augmenting, or generating messages on a communicator’s behalf—the review argues that algorithmic systems do not eliminate human intentionality but redistribute tasks across human and artificial entities. Evidence from multi-agent systems and bot networks indicates that campaigns can rely on semi-automated orchestration, delegating message testing, optimization, audience personalization, and community infiltration to generative tools. However, the author cautions that empirical evidence remains considerably stronger for assisted content creation and semi-automated workflows than for fully autonomous operations. While generative systems can simulate social dynamics and fabricate apparent consensus, claims of completely autonomous, end-to-end influence operations remain largely prospective, as human actors continue directing core campaign strategies and initiating operational goals.

The third strand of research synthesized in the review addresses the communicative vulnerability of audiences encountering increasingly realistic synthetic material. In experimental evaluations, Williams and colleagues tested 13 large language models, discovering that most systems broadly complied with requests to assist disinformation campaigns, could automate multiple sequential stages of an election influence operation, and generated outputs capable of passing as human writing. Complementing these textual capabilities, investigations by Gambín and colleagues and Yanes and colleagues show that deepfakes and multimodal synthetic media obscure the communicative markers historically used to evaluate information credibility. By combining scalable generation, micro-personalization, and human-like expression, generative artificial intelligence reduces the conventional cues that enable citizens to recognize deception. As communicative production and coordination become distributed across human coordinators and software models, authorial responsibility and deceptive intent grow increasingly opaque, elevating societal vulnerability and generating risks for public trust, opinion formation, and democratic discourse.

In its critical discussion, the review outlines three major conceptual challenges confronting sociological researchers analyzing online manipulation. The first challenge concerns the progressive redistribution of communicative agency across hybrid human-machine configurations. While modern artificial intelligence systems demonstrate the technical capacity to automate substantial portions of influence campaigns, Mannocci and colleagues note that empirical documentation of their extensive deployment in real-world operations remains limited. The review emphasizes that technological capabilities are currently advancing faster than empirical field documentation can verify them. From a sociological standpoint, generative artificial intelligence does not simply replace human actors; rather, human intentionality remains central while the execution of communicative labor is delegated to algorithms. This dynamic complicates conventional distinctions between human action and technological mediation. Consequently, researchers must move beyond asking who conducts manipulation to investigate how agency, intent, and accountability are distributed across socio-technical networks where human strategists and automated software systems interact.

The second and third conceptual challenges address the temporal persistence of campaigns and the criteria used to define communicative authenticity. Addressing temporality, the review references Lucas and colleagues to examine how generative disinformation creates long-term hazards for institutional trust and social cohesion, highlighting that coordinated influence can persist through the adaptation, recirculation, and reuse of synthetic material across campaigns. Regarding authenticity, the author distinguishes the authenticity of participation from human authorship, sincerity, or perceived genuineness. The review identifies two distinct mechanisms through which campaigns deceive audiences: rapid content production leaves users and platform moderators less time to verify whether an apparent consensus is genuine, while messages orchestrated by a single entity appear to originate from multiple independent speakers. Because speed and automation alone do not establish inauthenticity, the critical methodological challenge involves distinguishing evidence of strategic coordination from evidence that operational affiliations, origins, and relationships are actively concealed or deceptively represented to mislead observers.

Analyzing existing research gaps, the review emphasizes that scholarship remains fragmented across empirical domains and disciplinary traditions, with most studies concentrating narrowly on technical model features, detection strategies, or platform-specific manifestations. Crucially, the review notes that changes in perceived authenticity are not unique to manipulative contexts. In a vignette-based study of government-citizen interactions, Zhang and Nie found that artificial intelligence-assisted message revision improved citizens’ ratings of clarity, politeness, and the trust-building contribution of official responses. This demonstrates that altered perceptions of authenticity and relational communication occur across legitimate institutional environments as well as deceptive campaigns. The review highlights that institutions are not inherently exempt from manipulation, arguing that the essential analytical boundary separates coordinated deception about participation from computational mediation itself. Consequently, effective detection requires moving beyond algorithmic detection tools to develop operational criteria that rigorously distinguish coordinated collective activity from intentional deception regarding the organization and origins of online participation.

Beyond theoretical and methodological issues, the review underscores substantial societal risks, warning that human-like synthetic communication blurs boundaries between genuine human discourse and algorithmic generation, thereby complicating assessments of credibility and trust. Highlighting institutional warnings, the author notes that the United States National Institute of Standards and Technology cautioned in 2024 that generative artificial intelligence can facilitate large-scale disinformation, undermine information integrity, and complicate operational accountability. The review argues that persistent difficulties in detecting, attributing, and measuring coordinated inauthentic behavior must not be interpreted as evidence that generative artificial intelligence impacts are limited or purely hypothetical. Instead, methodological capabilities to quantify societal effects have simply lagged behind the rapid pace of technological diffusion. Responding effectively requires more than technical solutions; the review stresses the necessity of promoting public digital and artificial intelligence literacy that enables citizens to comprehend coordinated influence as an evolving socio-technical process, strengthening democratic resilience against deceptive manipulation.

In conclusion, the review warns that while coordinated inauthentic behavior poses severe risks to democratic discourse and public trust, researchers must reject technological determinism, recognizing that advanced algorithmic capabilities do not automatically guarantee successful persuasion, flawless coordination, or predictable political impact. The author emphasizes that the scarcity of direct empirical documentation in field environments should not be mistaken for an absence of impact, given that only three years have passed since generative artificial intelligence became publicly widespread. Because covert, distributed influence campaigns are intrinsically difficult to trace, detect, and quantify, generative manipulation may already be generating consequences that current measurement methodologies only partially document. Addressing these challenges requires sustained sociological research into hybrid human-machine agency and coordinated communication. The author concludes that mitigating these risks depends on developing interdisciplinary measurement frameworks that connect computational tools with social theories of persuasion, alongside fostering broader public awareness of how digital manipulation campaigns are organized, produced, and deployed.

Subject of Research: Sociological analysis of generative artificial intelligence and coordinated inauthentic behavior

Article Title: Coordinated online manipulation in the era of generative AI: emerging challenges

Article References: Murero, M. (2026). Coordinated online manipulation in the era of generative AI: emerging challenges. Frontiers in Sociology, 11, Article 1976855. https://doi.org/10.3389/fsoc.2026.1976855

Image Credits: AI Generated

DOI: 10.3389/fsoc.2026.1976855

Keywords: generative artificial intelligence, coordinated inauthentic behavior, digital manipulation, disinformation, sociology of communication, synthetic media, computational propaganda, democratic resilience, Coordinated, online, manipulation, generative