AI Summaries Can Rewrite What People Remember

Our memories aren't as fixed as we like to believe, and AI can alter them in surprising ways.

By now, we see all these AI summaries everywhere. They come up when you search something online, in meeting notes, medical records, police reports, and the apps we use every day. Like it or not, they’re everywhere.

It’s clear why they’re appealing: you save time by figuring out what happened and they’re super easy to make. But new research suggests that comes at a cost. They can change what we remember happened in the first place.

Don’t believe your eyes. Believe AI

The experiment started with a 25-second video. Or rather, two versions of a video.

Participants watched an animated scene of a red car approaching an intersection as a pedestrian stepped into the road. The car turned and hit the pedestrian. The pedestrian fell, got back up, and eventually met the driver.

Participants were showed one of two almost identical versions of this clip. In one, the intersection had a stop sign. In the other, it had a yield sign. That traffic sign difference was the key difference.

The setup is well known. It comes from a famous line of psychology research stretching back more than 50 years. Experiments using this type of setup have repeatedly shown that memory does not work like a video file stored in the brain. Basically, we reconstruct an event when we remember it — and information we encounter afterward can slip into that reconstruction.

Psychologists call this the misinformation effect. But what happens when AI comes into the mix?

“Smart” Summaries

After watching the accident video, participants left the experiment. They returned 24 to 48 hours later.

×

Get smarter every day...

Stay ahead with ZME Science and subscribe.

Daily Newsletter
The science you need to know, every weekday.

Weekly Newsletter
A week in science, all in one place. Sends every Sunday.
No spam, ever. Unsubscribe anytime. Review our Privacy Policy.

Thank you! One more thing...

Please check your inbox and confirm your subscription.

This time, they read a 21-sentence AI-generated summary of the event. For some participants, the summary correctly described the traffic sign they had seen. For others, the researchers switched it.

A person who had watched the car approach a stop sign might now read that it approached a yield sign.

Then the researchers gave participants a simple instruction: answer based on what you remember from the original video, not the summary.

The difference was enormous. Among people who read the accurate summary, 83.6% remembered the traffic sign correctly. Among those who read the misleading summary, just 44.8% did.

And this wasn’t a case of people just forgetting what happened in the video.

Participants got an average of 91.4% of the experiment’s other 11 memory questions right. For instance, more than 96% correctly remembered that the accident happened during the day — a detail the summary had not mentioned. Granted, it’s easier to remember day/night than the nature of a traffic sign, but all in all, participants’ memory was remarkably good.

Except where the AI told them something different.

Knowing it came from AI didn’t protect people

You might expect the solution to be simple, just tell people that the summary came from AI. Surely they’ll be skeptical and not trust it too much, right? Surely people will resist AI’s mistakes.

RelatedPosts

A Tiny Chip Placed Beneath AI Processors Could Cut Data Center Power Use by 50 percent
Cheap Robo-chef autonomously cooks a three-course Asian menu, cleans up after itself
The More Gen Z Uses AI, the More They Hate It
Making Mistakes while Learning facilitates Memory

Well, the researchers tested that, too, and that’s not how things panned out.

Before reading the summary, some participants were told it had been written by AI. Others were told it came from a professional human transcriber. In reality, the underlying text was AI-generated in both cases.

It barely even mattered.

The researchers also asked participants how much they trusted AI and how frequently they used AI chatbots. Again, they found no significant evidence that either protected people from the misleading summary.

It seems that simply knowing AI makes mistakes isn’t nearly enough.

“AI is a new method of delivering misinformation, and it has the potential to create these false memories for people who are reading that information,” said Mattea Sim, a study author, for a Georgetown University press release.

“I think we should be thinking really deeply and critically about whether and how AI should be used to summarize information, especially in high-stakes settings,” Sim added.

A bad summary can be dangerous without inventing anything

The researchers also tested how well the AI systems could summarize the videos in the first place. They asked ChatGPT-5.5 and Gemini 2.5 Flash-Lite to describe the same two 25-second accident clips, running each model five times on each video for a total of 20 summaries.

Every single summary contained mistakes. Some got details wrong or added things that never happened. But the most common failure was omission: on average, the models left out 51.6% of the event’s central details.

The most striking miss was also the most obvious one. Nineteen of the 20 summaries failed to mention that the car hit the pedestrian at all.

Hallucination is usually considered the main problem for AI failure. But this study points to a less spectacular but possibly more pervasive problem. Even when it’s not hallucinating, AI summaries can be misleading. And again, these summaries are already everywhere.

Of course, one could argue that these errors could creep up all the time, even before AI. But it’s not exactly the same thing.

A detail that disappears from a summary may become less available when a person later reconstructs an event. A detail that appears repeatedly may become easier to retrieve. A wrong detail may compete with the original one.

This is what makes AI summarization different from an ordinary transcription error. A transcription can be wrong on the page. A summary can become the story in your head.

As we increasingly use summaries precisely in situations where the original material is too long, tedious or complicated to revisit. We’re faced with a classic bargain: sacrifice detail for speed (or don’t).

At the end of the day, AI is already capable of providing speedy, convincing summaries. The new study reminds us that at least for now, this comes at a big cost.

The study was published in arXiv and has not yet been peer-reviewed.