Simulated Users and Virtual Labs: How Agent-Based Methods Are Rewiring Communication Research

Communication researchers are turning to a pair of computational techniques borrowed from software engineering and complexity science to study platforms whose inner workings have become nearly impossible to observe directly. A methodological review published in the journal Communication and Change argues that agent-based approaches, split into agent-based testing and agent-based modeling, offer a systematic framework for studying, explaining, and anticipating the societal consequences of algorithm-mediated communication, where content circulation and exposure are governed by continuously adaptive systems rather than stable, publicly accessible rules.

The review, authored by Wen Shi, Hanqing Zhao, Yangkun Huang, and Jinhui Li, describes what the authors call an observability crisis and a mechanistic dilemma. Algorithmic platforms deliver highly personalized recommendations built on digital traces such as browsing, clicking, and sharing, combined with social network relations and contextual cues. Yet even when platforms provide APIs, access is typically restrictive and shifting, and researchers cannot assume the returned data are comprehensive or representative of underlying user populations or content flows. Conventional empirical strategies can document patterned regularities and estimate effects, the authors write, but they struggle to specify how outcomes are endogenously generated through recursive interactions among users, algorithms, and media objects over time.

Traditional methods face specific failures in this environment, according to the review. Surveys and interviews suffer from recall bias, because users often cannot accurately remember all encountered content, especially in terms of frequency, sequence, or individualized recommendation details, creating substantial gaps between self-reports and actual exposure. Content analysis is distorted because algorithmically personalized feeds are private and generally inaccessible; even homepage snapshots or trending lists reflect only a fraction of public content and fail to represent the majority of individualized distribution patterns. Laboratory experiments allow variable control but rarely reproduce actual algorithmic feedback and dynamic evolution, since real-world platforms rely on instantaneous reactions and massive data adjustments while experiments are typically static and lack iterative updating.

Agent-based testing, or ABT, addresses the observability problem by deploying simulated user profiles on real platforms. The technique originated in computer science and software engineering as a system evaluation method, initially used for network service monitoring and automated software acceptance testing, before being adopted in communication research as an external auditing tool for social media platforms, news recommendation systems, and search engines. Its core concept is to simulate real users’ interaction paths and input conditions in order to quantitatively assess the performance of an algorithmic system without disrupting its normal functioning. Agents can be assigned explicit attributes such as age, interests, or political orientation, along with media usage habits and operational strategies like clicking, searching, or viewing, and they can interact with algorithms at high frequencies over extended periods, generating continuous and comparable data. Compared with real human users, agents have their attributes and behavior strategies precisely set and controlled, enabling participation in experiments involving sensitive topics or extreme situations without ethical risks.

The review identifies two broad directions in ABT applications: algorithms as objects and algorithms as environments. In the first, the algorithm itself is the primary object of investigation, auditing its operational logic and potential biases, such as whether platforms discriminate among users with different political orientations, genders, or geographical locations, or whether certain topics or sources are subjected to invisible practices like shadow banning. As an example, the authors cite previous studies showing that TikTok recommends a lower proportion of authoritative health information to users with inexpensive smartphones or in undeveloped regions, and that Google’s suicide prevention prompts respond unevenly to users of different languages. Such research typically employs highly controlled experimental designs to minimize confounding factors, producing precise measurements and causal inferences of algorithmic behaviors. In the second direction, algorithms are treated as integral components of the information environment, with studies combining longitudinal tracking and large-scale log analysis to examine how algorithmic distribution affects information diversity, whether personalization fosters news indifference, or whether it redirects audiences from news toward entertainment.

Agent-based modeling, or ABM, takes the complementary path. Originating in complex systems science, it applies computer-controlled simulation environments to model large numbers of heterogeneous agents and their micro-level interactions in a bottom-up manner, observing how nonlinear macro-level phenomena emerge, from social movements and community segregation to the evolution of traffic flows. Unlike ABT, which relies on real-world environments, ABM functions as a highly controllable, replicable, and intervenable virtual social laboratory. In communication research it has long been used for classic topics such as opinion dynamics and information diffusion, and in today’s algorithmically mediated media ecology it emphasizes theoretical exploration of how varying combinations of human-algorithm interactions and network structures may generate distinct communication outcomes, such as polarization, information diversity, or virality. Applications fall into two overlapping directions: theory testing, which translates abstract theoretical propositions into agent interaction rules to examine whether specific micro-level mechanisms are sufficient to generate observed macro-level phenomena, and policy simulation, which builds on validated models to evaluate the effectiveness of interventions such as algorithmic adjustments or content labeling across different scenarios, including which correction strategies are most effective in curbing rumor diffusion on misinformation-laden platforms.

Each method carries distinct strengths and limits. ABT offers high internal validity in real-world algorithm-mediated communication, allowing direct observation of how platform algorithms operate under rigorously controlled conditions without human participants, reducing costs, avoiding biases associated with self-reported data, and ensuring controllability and reproducibility. But its external validity is constrained by a limited design space: agents must operate within real platforms under tightly controlled behavioral scripts, and the underlying algorithms are fixed and opaque, their parameters and update rules cannot be systematically varied by researchers, making counterfactual exploration of alternative algorithmic logics virtually impossible. ABM, by contrast, excels at simulating complex dynamic systems and enables extensive scenario testing at no real-world cost, facilitating strong causal inference and theoretical development, yet it faces its own external validity challenges. Simulated models cannot perfectly capture real-world complexity, excessive simplification risks omitting crucial mechanisms while over-complexity can obscure causal pathways, and because ABM is highly sensitive to initial conditions, a small parameter change can yield substantially different outcomes, raising concerns about stability and reliability. ABM also requires calibration and validation against high-quality empirical data that is often constrained by platform policies and privacy protections.

The review’s central proposal is an iterative loop of measurement, modeling, prediction, and validation that links the two paradigms. In an illustrative pipeline examining how news diversity affects opinion polarization on TikTok, researchers would begin with an ABT experiment deploying automated user agents emulating various political leanings to quantify the proportion of like-minded versus cross-cutting news each agent encounters under the platform’s current recommendation logic. Those empirical distributions of exposure diversity and engagement probabilities are then converted into ABM parameters, for instance by using a diversity index to define the likelihood of cross-ideological content exposure. Next, an ABM simulation lets thousands of virtual agents interact within a TikTok-like recommendation ecosystem while researchers manipulate key algorithmic parameters such as the personalization coefficient or the weight assigned to engagement-based ranking, exploring counterfactual scenarios infeasible to test directly on the platform. Finally, simulation-based predictions guide new rounds of ABT validation, with experimental TikTok accounts deployed again to test whether hypothesized changes yield observable shifts in exposure diversity. The authors caution, however, that ABT-generated data are shaped by the commercial logic and technical architecture of specific platforms, and overfitting ABM models to such data risks replicating a platform’s idiosyncratic behaviors at the expense of theoretical abstraction and generalizability; they recommend treating ABT findings as heuristic evidence or stylized facts rather than fixed calibration constants.

A rapidly emerging frontier involves large language models, which push both methods toward what the authors call AI agents. Unlike rule-based counterparts, LLM-powered agents can process news texts and multimodal inputs including text, images, and audiovisual materials, generating contextually relevant responses akin to those of real users; they can autonomously adapt interaction strategies in response to system changes and algorithmic feedback rather than following rigid pre-defined rules; and given sufficient autonomy they may exhibit unanticipated strategies and collective adaptive phenomena. In ABT, this means virtual users evolve from predesigned scripts into adaptive entities that can autonomously detect sudden shifts in recommendation logic, interpret visual metaphors, affective tone, and narrative framing of videos, and dynamically update interest weights in response to ongoing exposure, forming historically dependent trajectories. In ABM, AI agents equipped with perception, memory, reasoning, and reflection transform simulations from static configurations into semi-autonomous evolution. The review cites pioneering work translating Maslow’s hierarchy of needs into computational motivations driving agent behavior, and studies using in-depth interviews with thousands of real individuals as the core memory for agents, achieving a transition from a general human simulator to a specific individual simulator. Standardized frameworks such as GovSim, which simulates common pool resource dilemmas, demonstrate how agents without preset rules can engage in natural language negotiation to persuade, compromise, or deceive, and simulations have shown AI agents reproducing collective phenomena such as the spontaneous emergence of information cocoons or quarantine fatigue during pandemics.

Yet the authors stress that incorporating AI agents does not eliminate inherent limitations and instead introduces new methodological and ethical challenges. The problem of explainability remains unresolved, since the internal decision-making processes of AI agents lack transparency, creating what the review describes as an even more sophisticated black box that undermines scientific credibility and replicability. LLMs carry risks of bias and hallucination, and their training corpora inevitably contain cultural biases, stereotypes, and factual inaccuracies that may be amplified within simulations, blurring the interpretation of emergent macro-level phenomena such as group polarization; researchers may struggle to determine whether such patterns result from designed social interaction mechanisms or from the intrinsic biases of the models. One cited characterization, the simulated uncanny valley phenomenon, holds that AI agents in ABMs are neither sufficiently abstract to clearly elucidate causal mechanisms as traditional models nor sufficiently natural to fully represent reality as humans, and even with LLM integration these models cannot fully replicate how real users exhibit emotional responses, trust dynamics, fatigue, and other psychological factors in human-algorithm interaction. Ethical concerns compound the methodological ones: AI agents placed in high-risk contexts involving violence, discrimination, or hate speech could generate content that violates ethical standards without clearly defined behavioral boundaries, and in ABT conducted on real-world platforms, large-scale, high-frequency automated interactions may be construed by platforms as disruptive activities, potentially resulting in account suspension, restricted data access, or even legal disputes. The review concludes that AI-driven ABM and ABT should be conceived not as complete replacements for traditional empirical approaches, but as complementary tools for probing theoretical mechanisms and predicting practical scenarios in an ever-evolving media landscape, with closer collaboration between researchers, platforms, and regulators needed to manage potential risks in real-world digital ecosystems.

Subject of Research: Agent-based testing and modeling methods for studying algorithm-mediated communication

Article Title: Agent-based approaches in studying algorithm-mediated communication: a methodological review

Article References: Shi, W., Zhao, H., Huang, Y., & Li, J. (2026). Agent-based approaches in studying algorithm-mediated communication: a methodological review. Communication and Change, 2(1), Article 2. https://doi.org/10.1007/s44382-026-00022-7

Image Credits: AI Generated

DOI: 10.1007/s44382-026-00022-7

Keywords: agent-based testing, agent-based modeling, algorithms, recommender systems, large language models, AI agents, communication research, TikTok, algorithmic bias, simulation, filter bubbles, research methods