Plate Nº 82 · recorded November 16, 2026
Earth & ClimateReported finding
AI Models Fold Under Pressure on Climate Facts, UNESCO-UCL Study Finds
A UNESCO-UCL study of 33 AI models finds accuracy on climate facts drops up to 35 percentage points when users push back, with policy analysis no better than chance.
By Nathan Brooks5 min read912 words
In brief
- Accuracy of 15 tested AI models dropped 31–35 percentage points when users repeatedly disputed correct climate answers
- Best models were 88–96% accurate on basic climate facts but only 71–77% on evaluating scientific claims
- On combined climate policy questions, AI accuracy was 33.8% — no better than random chance (33.3%)
- Accuracy fell from over 88% in English, Spanish, French and Portuguese to 78.3% in Swahili
- The report, led by UCL and published by UNESCO, launches at COP31 in Türkiye

Fifteen AI models lost between 31 and 35 percentage points of accuracy on climate questions when users repeatedly disputed their correct answers — and every single one eventually caved. That's the headline finding of a new UNESCO-UCL report, led by researchers at University College London, which will be launched at COP31, this year's United Nations climate summit in Türkiye.
The study examined 33 leading large language models — the AI systems behind chatbots — and tested five of them across 13 widely spoken languages, from English to Urdu. The picture it paints is mixed: strong on basic facts, shaky on reasoning, and weakest exactly where climate impacts hit hardest.
How accurate are AI models on climate basics?
When asked to provide basic climate change facts, the best-performing AI systems gave accurate information between 88% and 96% of the time, depending on the test. That's roughly nine correct answers out of every ten questions.
But ask the same models to judge whether specific evidence supports or refutes a scientific claim, and accuracy fell to between 71% and 77%. In other words, evaluation is harder than recall, and the numbers show it.
What happens when users push back?
The sharpest drop came when users expressed disagreement with factually correct answers. About 80% of AI concessions happened at the first sign of disagreement — one pushback, and the model often folded.
After being pushed several times, every model tested abandoned correct answers and adopted the user's incorrect biases. Researchers call this problem "AI sycophancy" — the tendency of a system to tell you what you want to hear rather than what's true.
Lead author Professor Ivana Drobnjak of UCL Computer Science, a member of the UNESCO Chair in AI at UCL, put it bluntly: "While AI can help to make basic facts about climate change easier to understand in some circumstances, our research shows that there are clear limits to its abilities."
"It is not a dependable substitute for scientists and qualified experts," Drobnjak said. "Its accessibility and conversational tone can give it an air of authority, but it cannot yet do the harder reasoning the climate problem demands, or hold on to a correct answer when a user pushes back."
Her prescription: "If AI is to help with climate change, it needs expert users who know how to question and check it, and AI systems built and tested specifically for the job."
Does accuracy change by language?
Yes, and significantly. In the five models tested across languages, responses were more than 88% accurate in English, Spanish, French and Portuguese. Accuracy then declined for languages with less digital text available online:
- Bengali and Hindi: 83.6%
- Urdu: 81.4%
- Swahili: 78.3%
That gap means AI models often delivered contradictory answers to the same scientific question depending on the language it was asked in. The researchers warn the problem cuts two ways: many of the communities facing the most severe climate impacts speak the languages AI handles least well. And reliability may be even lower in languages with less digital text than those tested.
Can AI evaluate climate policies?
This is where the models fell apart. Asked to assess how a single policy — say, a tax on coal — would affect global temperature by 2100, the AI models got it right up to 94.5% of the time.
But combining two policies changed the picture completely. Asked whether a coal tax plus a natural gas tax would yield more or less temperature reduction than the sum of each alone, the models answered correctly only 33.8% of the time on average. That's no better than random chance, which sits at 33.3% in this test.
To check the answers, the team compared AI responses against the En-ROADS climate simulator, a tool developed by Climate Interactive and MIT Sloan — the Massachusetts Institute of Technology's business school — that policymakers and climate educators widely use to explore policy and technology effects.
Why does fluent AI feel more trustworthy than it is?
Senior author Dr. Maria Perez-Ortiz of UCL Computer Science said: "AI systems are remarkably good at explaining scientific knowledge, but this can give us a false sense of their capabilities. Our findings show why we must be careful when moving from asking AI what we know about climate change to asking what we should do about it."
Co-author Leona Verdadero, a program specialist in UNESCO's Digital Policies and Digital Transformation Section, drew the line between recall and analysis. "Recalling facts is not the same as being able to provide reliable policy analysis," she said.
"The tendency for AI to misleadingly adapt answers to suit a user's point of view, rely on weak evidence or present speculation as analysis shows its limitations as a trustworthy broker of climate information to the public," Verdadero said. "As more people come to rely on it as a source of information, it's vital we understand what its shortcomings are."
The research was carried out by the UNESCO Chair in Artificial Intelligence at UCL, a team focused on promoting ethical AI development, policy and education globally. The report was commissioned to contribute to UNESCO's Global Initiative for Information Integrity on Climate Change. Its full title: "Can We Trust AI on Climate? Why it seeks to please the user, and fails on complex reasoning and multilingualism," authored by Ivana Drobnjak, Leona Verdadero, Zekun Wu, Byambaa Bayarmandakh, Wietske Holwerda, Sindi Gjonaj and Maria Perez-Ortiz.
via phys.org/rss-feed/ (Source)
More from Nathan Brooks
Show full bio
Market editor covering companies, deals and strategy in general science news at SciBeat.
402 articles