“Misinformation by Machine: The Rise of LLM Grooming in AI Systems”

Artificial intelligence has now become a part of the lives of researchers, writers, officials and general people, impacting their decisions and planning. But what if these LLM models, such as ChatGPT, Claude, Gemini, etc., are being massively fed with wrong information and leading the users in a direction not meant for them?

#ResponsibleAI #LLMGrooming #AITrust #misinformation #Disinformation

Artificial intelligence has now become a part of the lives of researchers, writers, officials and general people, impacting their decisions and planning. But what if these LLM models, such as ChatGPT, Claude, Gemini, etc., are being massively fed with wrong information and leading the users in a direction not meant for them? What if they are the propagators of disinformation? Yes, we are going to discuss the emerging threat of LLM Grooming.

---

The term "LLM grooming" was brought to light by The American Sunlight Project (ASP) [1]. In a report, the term was introduced, which describes how disinformation is being propagated by AI. These algorithmic machines are being trained massively behind the curtain to change the perception of their users about some political propaganda. Researchers from the ASP identified and investigated a network of pro-Russia propaganda websites named the Pravda network. A key finding of their research was the belief that the Pravda network may have been custom-built to flood large language models (LLMs) with pro-Russia content.

In a special reality check report released by NewsGuard [2] recently in March 2025, it has been confirmed that a well-funded, Moscow-based "news" network, named Pravda (Russian for "truth"), has been deliberately infecting Western artificial intelligence (AI) tools with Russian propaganda. They are not directly influencing human readers; they are manipulating around 10 leading generative AI models to achieve their disinformation goals by saturating search results and web crawlers with pro-Kremlin falsehoods in a significant percentage of their responses.

The Atlantic Council's DFRLab also confirmed that major AI models have cited Pravda network content.

What is LLM Grooming?

LLM grooming refers to intentionally manipulating the training data consumed by large language models (LLMs) such as ChatGPT, Claude, Gemini, etc. It is a technique whereby malicious actors feed misleading or agenda-driven content, specifically crafted to be ingested by large language models during their training cycles. They do not reach the public directly but hit the machine algorithms by feeding articles designed explicitly for scrapability, keyword density, and repeatability—not readability.

The European Union and organisations like the American Sunlight Project and EU DisinfoLab have identified LLM grooming as a new form of FIMI (Foreign Information Manipulation and Interference). Unlike the traditional strategies of spreading misinformation, this strategy operates at the machine level without even being known to the public, leading to disinformation and manipulating decision-making.

Real danger lies in the fact that these are machines. They cannot distinguish between verified information and well-disguised propaganda, and continue treating all the information as fact. Moreover, this happens very silently without the users even knowing.

---

Negative Impacts on Users:

For the average AI user—a student, researcher, policy analyst, healthcare professional, or journalist—the risks are far-reaching:

• Subtle Misinformation: In a very subtle way, misinformation is spreading among users, who may unknowingly rely on it—violating their right to accurate information.
• Weakness in AI: AI cannot distinguish between reliable and unreliable sources without security features that prevent or deter access to harmful websites.
• False Confidence: AI systems generate very polished and impactful responses, gaining users' confidence that the information being generated is correct. Moreover, when all the models are flooded by the same type of agenda-driven misinformation, users are compelled to believe it.
• Biased Decision-Making: As the AI models are used by individuals in high-stakes fields like finance, law, or public policy, even minor distortions in model reasoning can lead to poor outcomes.
• Feedback Loops: As the AI models are trained on the information that we feed them, and if they are ingesting disinformation, then they will present the disinformation to the users also, which will create feedback loops of garbage in-garbage out.
• Erosion of Trust: Once users realise that AI outputs might be groomed, this will lead to a lack of trust in AI models, resulting in the collapse of these models.

Possible measures can be taken to handle it:

• Rigorous Data Filtration: AI developers must clean their training data and avoid using known disinformation sources. Regularly monitor what data gets into your model. Checking domains, validating sources of information, and adding fact-checking layers are filters for models like  ClaimBuster and models like TrustworthyQA. [3] 
• Training Transparency: Lawmakers should mandate transparency and labelling for AI-generated content and ask AI developers to disclose what data sources were used to train AI systems.
• Public Awareness & Digital Literacy: Spread awareness among users about the cross-checking of information generated by AI, particularly on sensitive or news-related topics and also to identify and avoid unreliable sources. 

     LLM grooming is a growing technical challenge for AI developers, regulators, governments, policy makers, educators, researchers, and people in the health and economic sectors. The government should initiate national information-literacy programs to help adults and kids understand what they see online. At least the normal user must be aware if they are finding the repeated patterns of a key phrase. Models from different developers: GPT, Claude, Gemini, all responding with one narrative — it means they have been manipulated for presenting disinformation.

Overall, in an era of AI where the facts are increasingly shaped by these algorithmic machines, awareness might be our last defence. [4] We need to use generative AI with a high sense of responsibility by learning prompt engineering (how to ask questions to AI?) and critically assessing the generated responses.

Glossary

Misinformation

False or inaccurate information—getting the facts wrong.

Disinformation

False information which is deliberately intended to mislead—intentionally misstating the facts.

Source: American Psychological Association

References

1. 1. Resilience Media. (2025, April). When propaganda trains the bots: Why you should read about LLM grooming. https://www.resiliencemedia.co
2. 2. Constantino, T. (2025, March 10). Russian propaganda has now infected Western AI chatbots — New study. Forbes. https://www.forbes.com
3. 3. FactCheck.by. (n.d.). LLM grooming. https://factcheck.by/eng/news/llm-grooming/
4. 4. EU DisinfoLab & American Sunlight Project Webinar – Freuden, S. (2025, April 10). LLM grooming: A new strategy to weaponise AI for FIMI purposes. Webinar Details, YouTube Link
5. 5. LinkedIn – Bedassee, H. (2025). The silent threat shaping AI behind the scenes. LinkedIn Article
6. 6. The Washington Post. (2025, April 17). Russia seeds chatbots with lies. Any bad actor could game AI the same way. https://www.washingtonpost.com/technology/2025/04/17/llm-poisoning-grooming-chatbots-russia/
7. 7. Bulletin of the Atomic Scientists. (2025, March). Russian networks flood the Internet with propaganda, aiming to corrupt AI chatbots. https://thebulletin.org/2025/03/russian-networks-flood-the-internet-with-propaganda-aiming-to-corrupt-ai-chatbots/