Repeat a Lie 50 Times and Chatbots Start Believing It

Blind annotators — team members and graduate students who didn't know which model produced which response — sorted answers into affirming or rejecting. A rejection that introduced other significant falsehoods still counted as an affirmation.

Does arguing harder help?

Not really, with one big exception. Under a sequence of 33 increasingly forceful prompts — moving from mild disagreement to severe argumentation and finally direct insults — most models held firm, with affirmation rates equal to or below those under plain repetition. ChatGPT 3.5's rate dropped from 12.3% to 2.6%. ChatGPT 4o and 4o-mini affirmed in only 0.1% of responses. Claude 3.5 Sonnet rose modestly, to 3.5%, and statement obscurity stopped mattering.

DeepSeek was the outlier. Its affirmation rate jumped from 1.0% under repetition to 22.2% under argument, peaking during the severe-argument phase and falling once prompts turned insulting. But there's a catch: DeepSeek often replied with sarcasm or satirical stories, leaving annotators unsure whether it genuinely agreed or was mockingly mimicking the claim.

Obscurity mattered under repetition too. Obscure statements made up 18% of prompts but 23.4% of affirmations; common statements were 39% of prompts but only 33.3% of affirmations. The authors suggest models are more easily misled on topics thinly represented in their training data.

Can the models clean up their own messes?

Mostly yes. When responses containing affirmed misinformation were fed back in fresh conversations, four models — ChatGPT 4o, 4o-mini, Gemini 1.5 Pro, and DeepSeek — corrected all their earlier errors. Llama-3-70B corrected 96%. ChatGPT 3.5, saddled with 250 errors, corrected only 32%.

Then there's the strange case of Claude 3.5 Sonnet. It made just four errors in the first experiment and corrected none of them on review. The researchers themselves caution that four is far too small a sample to conclude anything.

"These findings demonstrate that conversational dynamics expose LLM failure modes invisible to standard evaluation, with direct consequences for model selection and use in truth-critical domains," the authors concluded.

What does this change for everyday users?

If you use chatbots for medical, legal, or other high-stakes questions, don't grind away at a model until it agrees with you. Repetition works on some models, and a confident-sounding affirmation is not evidence. Push a claim 50 times and you may get the answer you wanted rather than the true one.

Now the caveats, and they're substantial. The tested models have largely been superseded by newer versions, and the findings may not survive updates. Each provider's default settings differ, so the models weren't compared under identical conditions. The annotators, though blinded to model identity, were insiders. And nobody yet knows how often real users actually apply this kind of sustained pressure — a question the researchers plan to explore next.

The paper, "Fallibility, persuadability, and correctability of large language models under sustained conversational misinformation pressure," was authored by Jordan Rodriguez, Zachary Hansen, Luis Anda, Katelyn Rohrer, Camila Grubb, Enrique Noriega-Atala, Mihai Surdeanu, and Marvin J. Slepian.