A BBC investigation reveals that major AI chatbots, including ChatGPT and Gemini, struggle with accurately summarizing news, often producing misleading or incorrect information.

AI Chatbots Fail to Accurately Summarize News, BBC Study Reveals


Artificial intelligence chatbots are struggling to provide accurate news summaries, with a BBC investigation uncovering widespread inaccuracies in responses generated by leading AI models. The research, which examined OpenAI's ChatGPT, Microsoft's Copilot, Google's Gemini, and Perplexity AI, found that these tools frequently misrepresented or distorted key facts.

The study, conducted in December 2024, assessed how these AI models processed and summarized content from the BBC’s website. The findings raise serious concerns about the reliability of AI-generated news and its potential impact on public discourse.

Major Errors and Distortions in AI-Generated Summaries


BBC journalists tested 100 news articles, posing questions to the four AI chatbots and evaluating the quality of their responses. The results were striking: 51% of AI-generated answers contained significant issues, including factual inaccuracies and misleading claims.

A particularly concerning discovery was that 19% of responses citing BBC content contained errors, such as incorrect statistics, misattributed quotes, and inaccurate timelines. Among the most glaring mistakes:


  • Google’s Gemini incorrectly stated that the NHS does not recommend vaping as a smoking cessation aid.

  • ChatGPT and Copilot erroneously claimed that Rishi Sunak and Nicola Sturgeon were still in office after they had stepped down.

  • Perplexity AI misquoted BBC News in a report on the Middle East, falsely stating that Iran initially showed “restraint” and that Israel's actions were “aggressive.”

BBC News Director Deborah Turness warned that such errors could have real-world consequences, emphasizing that AI-generated misinformation poses a growing risk in an era of political and social turbulence.

Calls for Greater AI Accountability


In response to the findings, Turness called on AI developers to take urgent action, warning that companies behind these chatbots were “playing with fire.” She urged tech firms to collaborate with media organizations to improve accuracy, stating that the BBC is ready to engage in discussions to address the issue.

Turness also noted that Apple had previously pulled its AI-generated news summaries following complaints from the BBC about misinterpretations. She encouraged other companies to follow suit and reassess how their AI tools process and summarize news.

While the BBC typically blocks AI crawlers from accessing its content, it temporarily lifted restrictions for this research. The study’s findings highlight the pressing need for AI developers to refine their models and ensure that users receive accurate and trustworthy information.

As AI continues to play an expanding role in news consumption, the BBC’s investigation underscores the importance of responsible AI development and the potential dangers of unchecked misinformation.