Heavy AI Use Blurs Human Detection of Fake Content, Training Restores It

Generative artificial intelligence has become so convincing that the line between what people create and what machines produce is increasingly difficult to see. A new study published in AI & Society by researchers at Temple University, led by Tanaka Manhede and Jason Chein, set out to answer two pressing questions: does everyday experience with AI tools change a person’s ability to tell real content from synthetic content, and can that ability be deliberately improved? The findings are both cautionary and hopeful. People who engage frequently and variedly with AI platforms were significantly worse at spotting AI-generated material, yet a relatively brief, structured training intervention lifted detection accuracy by nearly ten percentage points, while untrained participants showed no improvement at all.

The concern motivating the research is well founded. Generative AI systems now produce text, images, and videos that closely mimic human work, and these outputs have already been exploited in disinformation campaigns, AI-written phishing emails, deceptive voice cloning, and fabricated hotel reviews. Prior studies have found that human evaluators perform at or near chance when judging whether scientific abstracts, faces, or videos are authentic. Some evidence suggests that expertise helps: professional writers detect AI-authored essays better than lay readers, and people judge general-interest news more accurately than scientific news, presumably because they lack domain knowledge to spot subtle errors. Multimodal information, such as combining transcripts with audio and video, also improves detection, hinting that a real, detectable signal separates human from machine output even when average performance is poor.

The Temple team hypothesized that individual differences in attitudes toward AI, and in the history of engagement with AI tools, might explain why some people discern better than others. Two competing predictions framed the question of AI use. On one hand, frequent use of a platform could act as practice, sharpening sensitivity to that system’s stylistic fingerprints; one recent study found that heavy ChatGPT users were indeed better detectors of ChatGPT-written text. On the other hand, theories of extended cognition and cognitive offloading suggest that habitual reliance on external tools to support reasoning may blur the boundary between information arising inside one’s own mind and information arriving from outside. Under that view, heavy AI engagement could desensitize users to the very cues that distinguish synthetic from human content.

To test these ideas, the researchers recruited 117 adults aged 18 to 34 through the Prolific platform, all fluent English speakers located in the United States. The stimuli were 176 vacation rental listings presented in an Airbnb-like format. Half were genuine: descriptions extracted from real Airbnb listings with at least ten public reviews and ratings of 4.5 stars or higher, verified as human-written with an AI detector, and paired where appropriate with a real photograph of the property and a real human face drawn from the Flickr-Faces-HQ dataset. The other half were synthetic: texts generated by ChatGPT 4.0 and Claude 3.5 Sonnet describing fictional rentals, matched to the human texts in word count, and accompanied by AI-generated property images from Mage.space and Gemini plus AI faces produced by StyleGAN2. Notably, the AI texts were actually grammatically superior to the human ones, ruling out simple error-spotting as a strategy.

Participants first completed a baseline assessment, judging 32 listings as human or AI generated, with half presented as text only and half as text plus images. They then moved through two intervention or control stages, with assessments interleaved, before finishing with questionnaires measuring AI attitudes on the ATTARI-12 scale and AI engagement on a newly developed survey called the Socioaffective and Cognitive Artificial Intelligence Engagement Survey, or SCAIES. At baseline, participants were 57.4 percent accurate on average, only modestly above chance, but individual scores ranged from 34 to 90 percent, a spread that invited explanation.

The explanation that emerged was striking. Attitudes toward AI, which were generally positive in this sample, did not predict detection performance at all. But SCAIES engagement scores did: heavier overall engagement with AI was associated with significantly weaker discernment, a correlation of negative 0.29. The same negative relationship held separately for socioaffective uses of AI, such as social and emotional purposes, and for cognitive uses, such as outsourcing effortful mental tasks. Interestingly, an objective measure of ChatGPT usage, drawn from participants’ logged session counts over the prior 30 days, did not predict discernment, even though participants slightly underestimated their actual use. This dissociation suggests that it is not raw frequency of use but the qualitative breadth and motivational depth of AI integration into daily thinking that erodes sensitivity to the difference between synthetic and human content, consistent with the idea that deeply enmeshing AI into one’s cognitive life makes machine output feel less foreign.

The intervention half of the study offered a counterweight to this bleak picture. Sixty participants were assigned to an experimental group and 57 to a control group, with the two groups performing identically at baseline, around 57 percent accuracy. The experimental group first viewed 40 accurately labeled human and AI exemplars, half text-only and half multifeatured, with a minimum ten-second viewing period per item. The control group viewed the same number of unlabeled listings for the same duration. In the second stage, experimental participants judged 40 new listings while receiving immediate Correct or Incorrect feedback and earning 25 cents per correct answer, with cumulative earnings displayed on screen. Control participants received the same monetary incentive but only delayed feedback and no running tally. The results were decisive: the experimental group improved by 9.4 percentage points from baseline to final assessment, while the control group changed by a negligible negative 0.6 points.

Most of the gain, 6.7 percentage points, came after the labeled exemplar stage alone, indicating that detection failure stems less from an absence of diagnostic cues than from uncertainty about which cues matter. Explicit labels appear to recalibrate attention toward informative properties. The feedback and incentive stage added a further, non-significant 2.7 points. Reaction time analyses ruled out a speed-accuracy tradeoff, showing that improved performance reflected more efficient use of cues rather than slower deliberation. Encouragingly, participants with the lowest baseline scores gained the most, and prior AI engagement, which predicted poor baseline performance, did not predict resistance to training. Any desensitization caused by habitual AI use appears reversible with structured exposure and feedback.

Item-level analyses revealed what cues successful detectors may have exploited. Using Google’s Universal Sentence Encoder to compute semantic similarity between all pairs of texts, the researchers found that AI-generated listings were significantly more similar to one another than human-generated listings were, with intra-class similarity of 0.50 for AI texts versus 0.45 for human texts. In other words, AI outputs converge toward prototypical patterns while human writing displays greater idiosyncratic variety. Human texts that stood out as most semantically distinct were judged most accurately at baseline, suggesting evaluators are implicitly sensitive to this distributional signature. Parallel image analyses using a ResNet-50 model showed the same pattern of greater homogenization among AI-generated images. Participants were also better at identifying AI texts in the text-only condition but better at identifying human listings when images were present, implying a shift in cue use across modalities.

The broader implications reach into education, digital literacy, and information integrity. As generative AI embeds itself in everyday tasks, spontaneous sensitivity to its distinguishing qualities may quietly diminish, a paradox the authors describe as normalization. If intuitive detection erodes, intentional training frameworks and accurate labeling of AI content may become essential safeguards. The study also carries a caveat: current generative systems still leave detectable statistical regularities, but as models evolve and diversify, those signatures may fade. The stimulus set was limited to promotional vacation rental language, the sample was restricted to young, digitally fluent adults, and the durability of training gains remains unknown. Still, the central message stands: human discernment of AI content is neither obsolete nor fixed. It varies widely across individuals, is dulled by habitual AI engagement, and can be measurably restored through brief, well-designed training, offering a practical path toward keeping human judgment sharp in a marketplace increasingly saturated with synthetic content.

Subject of Research: How experience with AI tools and targeted training influence human ability to distinguish AI-generated from human-created online content

Article Title: Human discernment of artificial intelligence in online markets can be shaped by experience and training

Article References: Human discernment of artificial intelligence in online markets can be shaped by experience and training. (n.d.). https://doi.org/10.1007/s00146-026-03373-3

Image Credits: AI Generated

DOI: 10.1007/s00146-026-03373-3

Keywords: artificial intelligence, AI detection, generative AI, digital literacy, cognitive training, human-AI interaction, misinformation, cognitive offloading, AI & Society, Temple University, synthetic content, extended cognition