AI Writing Tool Reshaped Petition Language on Change.org but Failed to Boost Success

When the petition platform Change.org quietly rolled out a built-in artificial intelligence writing assistant, it offered millions of would-be activists a tempting promise: a smoother, more polished appeal that might finally tip the scales in their favor. A new study published in Nature Human Behaviour suggests the promise was only half kept. Researchers led by Isabel Corpus of Cornell University, together with Eric Gilbert of the University of Michigan, Allison Koenecke of Cornell, and Mor Naaman of Cornell and Cornell Tech, found that the tool dramatically changed how petitions were written, yet left the ultimate fate of those petitions essentially untouched. The work offers one of the clearest real-world measurements to date of what happens when generative AI is embedded directly into a large online platform, and its central finding is a sobering one for anyone hoping that better-sounding text translates into better results.

The study’s strength lies in its design. Rather than running a small laboratory experiment, the team exploited what statisticians call a natural experiment: Change.org introduced its ‘write with AI’ feature in some countries before others, creating a clean before-and-after boundary. The researchers collected roughly 1.5 million petitions and applied a difference-in-differences analysis, a causal inference technique that compares changes over time between groups that received the AI tool and groups that did not. This approach, widely used in economics and built on methods formalized by Callaway and Sant’Anna for settings with multiple time periods, allows researchers to isolate the effect of the intervention from broader trends that would have happened anyway. Because the rollout timing was determined by the platform rather than by users’ characteristics, the comparison approximates the rigor of a randomized controlled trial at population scale.

The first major finding concerns the texture of the text itself. After the AI tool became available, the lexical features of petitions shifted measurably at the platform level. Petitions grew longer and their language changed in ways consistent with the stylistic fingerprints of large language models: smoother phrasing, more standardized vocabulary, and altered readability characteristics. These are not trivial cosmetic shifts. Lexical measures such as vocabulary diversity and readability scores have long been used in computational text analysis to characterize how writing differs across authors, genres, and eras, and the fact that a single platform feature could move these metrics across an entire ecosystem of user-generated content is striking. In effect, the introduction of the tool bent the collective writing style of a global advocacy community.

But the second major finding is where the story turns. None of these stylistic improvements translated into better petition outcomes. Petitions written with access to the AI assistant did not attract more signatures, achieve their goals more often, or otherwise perform better than petitions written without it. For a tool presumably offered to help advocates succeed, that null result carries real weight. It echoes a growing body of evidence that generative AI can boost individual productivity in controlled settings, as shown in the well-known 2023 Science experiment by Noy and Zhang on professional writing tasks, while failing to deliver equivalent gains when success depends on persuading other humans in a noisy, competitive environment.

The researchers did not stop at the platform-level analysis. A persistent worry with observational studies is that the people who choose to use a new feature differ systematically from those who do not, contaminating any comparison. To address this, the team conducted a separate analysis of repeat petition writers: 4,611 users who had written at least one petition before the AI tool was introduced and another one after. This within-person comparison holds individual ability, motivation, and topic preferences roughly constant, isolating the effect of AI access on the same writers over time. The pattern that emerged was telling. On average, these returning writers produced longer petitions when they had access to AI, yet their second petitions fared worse in terms of outcomes than their earlier work. The very users most likely to benefit from accumulated platform experience saw no advantage, and by some measures a disadvantage, from the writing assistant.

Perhaps the most consequential finding is the increase in homogeneity. Using dynamic difference-in-differences estimates, the researchers showed that platform-wide homogeneity among petitions rose after the AI tool’s introduction. In plain terms, petitions began to sound more like one another. This result aligns with a rapidly expanding literature on what some researchers call generative monoculture. A 2024 study in Science Advances by Doshi and Hauser found that generative AI enhanced individual creativity while reducing the collective diversity of novel content. Work presented at the 2025 CHI Conference by Agarwal, Naaman, and Vashistha showed that AI writing suggestions can homogenize text toward Western stylistic conventions and erode cultural nuance. Padmakumar and He asked directly whether writing with language models reduces content diversity and found evidence that it does. The Change.org study extends these laboratory and small-scale findings to a genuine production environment used by millions.

Why should homogenization matter for something as seemingly benign as petition language? Advocacy platforms depend on differentiation. A petition competes for attention in a crowded feed, and its distinctive voice, urgency, and local specificity are part of what makes a reader stop and sign. When an algorithmic assistant nudges thousands of writers toward the same polished register, the collective signal degrades even if each individual text improves. Researchers have raised parallel concerns in other domains: a 2026 Nature analysis by Hao and colleagues found that AI tools expanded individual scientists’ output while contracting the focus of science as a whole, and work on Stack Overflow suggested that large language models may threaten the value of community-generated digital public goods. The pattern that emerges across these studies is consistent: individual-level gains, community-level costs.

The null effect on outcomes also invites a deeper question about what petition success actually depends on. Prior research on e-petitions, including studies of linguistic cues and multidimensional time-series predictors of success, has suggested that factors such as topic salience, timing, social networks, and media coverage often dwarf the influence of prose quality. A beautifully written petition about an issue nobody cares about will still fail; a clumsy petition riding a wave of public outrage may succeed regardless. The Change.org results are consistent with this view. If outcomes are driven primarily by the underlying demand for a cause rather than by the craft of the appeal, then a tool that polishes craft without touching demand should indeed leave outcomes unchanged. The study cannot rule out subtler possibilities, such as offsetting effects in which longer, more polished text helps in some respects and hurts in others, but the headline conclusion stands: no measurable outcome benefit at scale.

There are also social and perceptual dimensions that the authors situate within a broader research program. Studies have shown that human heuristics for detecting AI-generated language are unreliable, that there can be a social evaluation penalty for using AI, and that perceptions of AI-mediated communication affect trustworthiness and perceived authenticity. For petition signers, the suspicion that a heartfelt appeal was machine-generated could matter as much as the text itself. Related work on the ‘AI ghostwriter effect’ shows that users often fail to feel ownership of AI-generated text while still claiming authorship, raising questions about authenticity in civic expression. A platform that encourages AI-assisted advocacy may therefore be trading not only diversity of style but also the perceived sincerity that makes grassroots appeals compelling.

The practical implications reach well beyond one platform. Companies including Amazon and Meta have embedded generative writing tools into seller listings and social posts, making the Change.org experiment a preview of dynamics likely to unfold across the internet. The study’s message for platform designers is that embedding AI writing assistance is not a neutral upgrade: it reshapes the linguistic character of an entire content ecosystem and can introduce unintended consequences such as homogenization without delivering the hoped-for performance gains. For researchers, the work demonstrates the power of natural experiments and difference-in-differences designs for evaluating AI interventions in the wild, complementing randomized trials that capture only narrow populations and short horizons. And for the millions of people who turn to online platforms to press for change, the findings carry a quietly empowering lesson: the cause, the community, and the timing still matter more than the polish of the prose. The data and code behind the analysis have been released through the Open Science Framework, allowing other researchers to reproduce the results and probe further into how generative AI is quietly rewriting the way the public speaks online.

Subject of Research: Causal effects of an in-platform AI writing tool on petition content and outcomes on Change.org

Article Title: Introducing AI to an online petition platform changed outputs but not outcomes

Article References: Corpus, I., Gilbert, E., Koenecke, A., & Naaman, M. (2026). Introducing AI to an online petition platform changed outputs but not outcomes. Nature Human Behaviour. https://doi.org/10.1038/s41562-026-02580-8

Image Credits: AI Generated

DOI: 10.1038/s41562-026-02580-8

Keywords: generative AI, Change.org, online petitions, difference-in-differences, content homogenization, large language models, computational text analysis, Nature Human Behaviour, platform design, AI writing tools, natural experiment, digital activism