Misleading AI Summaries Can Rewrite What People Remember, Study Finds

Artificial intelligence systems are now woven into the everyday flow of information, producing summaries of news articles, workplace meetings, medical records and even video footage from police body cameras. A new study from researchers at Georgetown University and the University of Washington suggests that this convenience carries a hidden cost: when an AI system summarizes an event inaccurately, it can actively distort what people remember about that event, reshaping their perception of the truth. The research, titled “AI-Enabled Human Memory Manipulation: Misleading AI-Generated Summaries Distort Human Memory,” was published in September 2026 and will be presented at the Ninth AAAI/ACM Conference on AI, Ethics and Society, a peer-reviewed academic conference taking place in October 2026.

The study was led by Mattea Sim, an assistant research professor at the Massive Data Institute in Georgetown’s McCourt School of Public Policy. Her co-authors were Yoshi Kohno, the McDevitt Chair in Computer Science, Ethics, and Society and professor at Georgetown’s Department of Computer Science and Center for Digital Ethics, and Yael Eiger, a Ph.D. candidate at the University of Washington whose research focuses on technology in the carceral system. Kohno framed the work as part of a broader institutional effort, saying the study is part of a growing effort at Georgetown focused on exploring the impact and relationship between AIs and humans, grounded in both psychology and computer science.

The researchers adopted two distinct approaches to understand how AI summaries interact with human memory. In the first phase, they analyzed summary outputs produced by OpenAI’s ChatGPT and Google’s Gemini to quantify how frequently errors appeared and what categories those errors fell into. The results were sobering. The AI summaries frequently omitted critical details, misconstrued information and hallucinated inaccurate details that were not present in the underlying material at all. The source material was an animated video depicting a car and pedestrian accident, and the summaries were measured against the actual contents of that video to determine how faithfully the models reproduced what happened.

The most common category of error was omission, meaning the failure to mention the primary events of the animated video. Across all prompts and models, the summaries omitted 51.6 percent of central details on average. Even more striking, all but one of the summaries — 95 percent of them — omitted the single most central detail of the event: the fact that a car collided with a pedestrian. Eiger said she was struck by how bad the summaries were, even at this stage in AI development, and noted that it worries her that police departments may be using video summarization technologies without rigorous testing and without an awareness of how incorrect AI-generated summaries could be.

The second phase of the study examined what those errors do to the people who read them. Participants were first asked to watch one of two animated videos, which were created in a recent replication study that modernized materials from a classic human memory experiment. The researchers were careful about the nature of the footage. Kohno emphasized that they did not use actual police body camera footage. Instead, the videos depicted a red car approaching an intersection with either a stop or a yield sign, with participants randomly assigned to see one of the two traffic signs. In the videos, the car turns right at the intersection and collides with a pedestrian walking into the road. After being knocked down, the pedestrian stands up, the driver exits the car and the two meet.

Between 24 and 48 hours later, participants returned for the second portion of the experiment. Each was randomly assigned to read either an AI-generated summary that was accurate or one that contained an inaccurate detail changed by the researchers. Before reading, participants were also told that the summary had been written by either an AI system or a human transcriber. In reality, all of the summary text was generated by ChatGPT, and in some cases separate parts of the model’s answers were stitched together to ensure that the experimental conditions remained uniform across participants. In total, 331 people completed both parts of the experiment, providing a substantial sample for measuring how the summaries shaped recall.

The memory effects were dramatic. When participants were asked whether the car in the video had approached a stop sign or a yield sign, 83.6 percent of those who read a summary with accurate information answered correctly. Among those who read a misleading summary, only 44.8 percent were correct — a gap of nearly 39 percentage points. Participants who read a misleading AI summary were significantly less likely to accurately recall the original event compared to people who read an accurate summary. Crucially, the manipulation worked even when people knew the summary came from a machine. The impact of misleading AI summaries on memory did not depend on participants’ trust in AI or their familiarity with it, suggesting that susceptibility to this form of misinformation is broad and not easily guarded against by skepticism about the technology.

Sim described the phenomenon in stark terms, saying that AI is a new method of delivering misinformation and that it has the potential to create false memories for people who are reading that information. She argued that society should be thinking deeply and critically about whether and how AI should be used to summarize information, especially in high-stakes settings. The authors made a similar point in the study itself, writing that the findings have implications for how AI should be used in critical settings. They noted that although humans-in-the-loop are often expected to correct for AI’s mistakes, their work suggests human memory can instead be distorted by those mistakes, and that AI has the potential to generate misinformation even absent any adversarial intent, which can meaningfully impact human memory.

The implications extend well beyond the laboratory. As individuals and institutions grow more reliant on AI, Kohno hopes the findings encourage people to recognize that AI outputs can have a direct impact on how humans perceive the world. That concern carries particular weight in domains such as policing and medicine, where a distorted summary of an event could influence investigations, legal proceedings or clinical decisions. Eiger, whose research examines technologies increasingly implemented by police for surveillance or departmental efficiency, said these tools are often advertised and framed as beneficial, but the trade-offs remain poorly understood. She would like to see more audits of this type of technology and pointed out that, from a public transparency perspective, there is little insight into how the deployed systems work or which exact models are being used in the race to bring AI into every sector.

The authors also acknowledge the limits of the current work and point toward what comes next. The study used the most up-to-date consumer-facing versions of large language models, but the researchers did not have access to enterprise versions of AI designed for organizational use, which may behave differently. For future studies, the team wants to move beyond animated videos and examine real-world examples. Sim asked what happens with AI summaries of police body camera footage and what police reports look like when they are generated by AI, and whether such summaries manipulate people’s memories of police-civilian interactions and consequently their perceptions of what happened in an incident. Those questions, she suggested, would be very interesting to answer — and the answers may determine how safely AI summarization can be entrusted with society’s most consequential records.

Subject of Research: The effect of misleading AI-generated summaries on human memory and perception of truth

Article Title: New research finds that misleading AI-generated summaries can distort human memory

Article References: New research finds that misleading AI-generated summaries can distort human memory. (n.d.). Original publication

Image Credits: AI Generated

DOI: Not provided

Keywords: artificial intelligence, AI summaries, human memory, misinformation, ChatGPT, Gemini, large language models, memory distortion, police body camera footage, AI ethics, Georgetown University, University of Washington