A new report from GDI shows how such models are being exploited to disseminate Russian state narratives. In an era where zero click searches have reached 60%, whoever owns the AI summary box on search engines, controls the narrative.

The creation of false narratives has always been a tool of international political conflict: the Zinoviev letter in the 1920s in Russia, the Marco Polo Bridge incident in the 1930s in China and the Gulf of Tonkin incident in the 1960s in Vietnam are but three famous examples of invented stories used for disruption or to justify war. Now, Large Language Models from leading AI companies can be used to turbocharge such propaganda and amplify its effects — and our analysis shows that this is being done regularly by entities linked to the Russian state and others, frequently without being detected, rebutted or placed into context. Such narratives risk becoming widely believed to be true, thanks to the apparent endorsement of artificial intelligence. However the benefits of technology can work both ways: Users and the LLM-providers could take measures to reduce or even eliminate this risk. Our analysis recommends one potential approach.

The two models reviewed in this study are Claude Sonnet and Mistral Large. Both were found on occasion to repeat Kremlin narratives “in their own voice” - with no context or rebuttal, making it hard for a reader to spot.