AI Porkies

It’s good to have your own experience validated by research and the Columbia Journalism Review quotes a study by the Tow Centre for Digital Journalism which has found a substantial proportion of inaccuracies in answers to questions put to generative AI search engines,.

The study tested eight generative search tools: OpenAI’s ChatGPT Search, Perplexity, Perplexity Pro, DeepSeek Search, Microsoft’s Copilot, xAI’s Grok-and Grok-3 (beta), and Google’s Gemini.

Collectively, the tools provided incorrect answers to more than 60% of queries.

Perplexity answered 37% of the queries incorrectly, while Grok 3 answered 94% of the queries incorrectly.

And don’t go paying for premium versions of these AI tools because they were found by Tow’s research to be less trustworthy in their answers than the free versions, possibly because they  tried to give definitive answers rather than honest answers when they were not sure of the right answer.

Where the AI search engines lag traditional search engines is that whereas  traditional search engines typically operate as an intermediary, guiding users to news websites and other quality content, generative search tools parse and repackage information themselves cutting off traffic flow to original sources. 

Tow found that:

Chatbots were generally bad at declining to answer questions they couldn’t answer accurately, offering incorrect or speculative answers instead.

Premium chatbots provided more confidently incorrect answers than their free counterparts.

Multiple chatbots seemed to bypass Robot Exclusion Protocol preferences.

Generative search tools fabricated links and cited syndicated and copied versions of articles.

Content licensing deals with news sources provided no guarantee of accurate citation in chatbot responses.

So, before going down the pub to drown your sorrows after a bot tells you that Vlad’s troops are on the way, check it out with a good old human being.