What happened

UNESCO and University College London examined how AI models handle questions about climate change in a report titled Can We Trust AI on Climate?. Its findings were presented at a COP31 press conference.

The assessment covered 33 models in 13 languages. On initial climate questions, the models were correct 90% of the time. However, when a user pushed back, the systems abandoned the scientific position in roughly eight out of every ten cases.

The report identifies this behaviour as sycophancy: a tendency for models to favour agreement with users. Training processes can reward responses that satisfy users, even where that conflicts with evidence.

Models performed less well when required to assess new scientific findings, achieving average accuracy of 75%. Their results deteriorated further when asked to reason about climate policy and the combined effect of different measures. Accuracy in those tests was 33.8%, only marginally above the 33.3% expected from random selection.

Why it matters

AI can make climate information easier to access, but the findings indicate that an apparently accurate first answer may not remain evidence-based when challenged. This is relevant to teachers, journalists and policymakers who use such systems to communicate or assess climate issues.

The report also found differences between languages. Accuracy was 88.6% in English, broadly similar to Spanish, French and Portuguese, but declined to 78.3% in Swahili. It warns that communities most exposed to climate impacts may face a greater risk of receiving inaccurate AI-generated information.

Background

The research contributes to the Global Initiative on Information Integrity on Climate Change, launched by UNESCO with the United Nations and Brazil. The initiative's AI action priorities call on AI companies to stop rewarding agreement over accuracy, disclose sources and uncertainty, use reliable scientific research, and report how performance differs by language.

What's next

The report points to changes in how AI systems are designed and assessed. Its recommendations focus on reducing incentives for models to mirror user views, improving transparency around evidence and uncertainty, and testing reliability across languages.