Artificial intelligence doesn’t have a great reputation for veracity. Social media abounds with AI-generated concoctions, from cute images of fake animals in fabricated settings to violent videos depicting imaginary destruction in the Middle East. AI has been used to create fake social media accounts that spew Russian government propaganda and to generate clickbait and misinformation for content farm sites that exist solely to collect ad revenue. In 2024, thousands of New Hampshire residents received robocalls from an AI-synthesized voice of Joe Biden that discouraged them from voting in primary elections.
With so much AI-generated rubbish threatening to blur the line between reality and fiction, it may seem counterintuitive that scientists are exploring ways to use the very same technology to combat online misinformation. But researchers are finding that AI’s ability to parse human language, summarize text and verify claims could be harnessed to help people identify and understand fake news — and perhaps even one day assist in combating online misinformation in a large-scale, systematic way.
In a world where online misinformation about current events has influenced elections and incited political violence, such tools could be invaluable for journalists, fact-checkers, social media companies and others who strive to rid the web of fake news.
Across a diverse range of nations, most adults say they see online misinformation as a major threat to their country, according to a recent survey by the Pew Research Center.
It’s still early days, and experts stress that these methods — like all AI tools — should never be used without some level of human supervision. But researchers also see AI as an important ally against fake and misleading news, even if it was AI that made the fake news in the first place. “We should fight fire with fire,” says Jevin West, an expert on misinformation and generative AI at the University of Washington.
True or false?
A relatively straightforward kind of AI, called machine learning, has long been used to identify falsehoods. This approach entails giving computational models claims that have been verified true or false by human fact-checkers. The models determine the textual features, patterns and phrases that correlate with the likelihood of a statement being bogus — say, overuse of capital letters, exclamation points or emotionally charged language — and use these characteristics to sort new inputs into true and false categories.
When researchers trained one such model to identify tweets containing Covid-19-related misinformation during the pandemic, “we were actually pretty efficient,” says data scientist Zois Boukouvalas of American University, who coauthored a review on misinformation detection by machine learning in the Annual Review of Statistics and Its Application. The program agreed with human fact-checkers roughly 90 percent of the time.
But because machine learning models have typically been trained on well-curated datasets that tend to capture only particular time periods, topics or social media platforms, they don’t have the flexibility to be useful in real-world situations, says Dorsaf Sallami, an AI and information integrity researcher at McGill University and Mila, an AI research institute, in Montreal. She and other researchers have turned to the large language models (LLMs) that power AI chatbots like ChatGPT, which are trained on massive amounts of public internet content, among other sources. Unlike machine learning models, they analyze the relationships between words, phrases, concepts and contexts to build knowledge about language patterns, which they can then use to generate new text. Another advantage is that many newer LLMs have built-in ways to also analyze image- and audio-based data.
Thanks to their deep understanding of human language, LLMs can help with analyzing claims, says AI research scientist Thanh Thi Nguyen of the University of the Sunshine Coast in Australia. Ask them whether something is true or not, and they reply based on the patterns they’ve learned from ingesting massive amounts of online information. But because they’re much more like language imitation machines than lie detectors, they can also produce false information, he says: When given ambiguous or insufficient information, they’re prone to confidently concocting their own misinformed responses in a process known as hallucination. In the context of misinformation, this issue partly arises because LLMs are not necessarily trained on the latest current events and not all can do live searches.
To contend with this issue, Sallami has developed a fact-checking browser extension that enables an LLM to search the web for up-to-date information before generating an answer (in the same way as many publicly available chatbots, such as Grok). But this strategy isn’t bulletproof. One preliminary study found that an early 2025 version of Grok, for instance, agreed with human fact-checkers only roughly 55 percent of the time when asked by users to verify claims. Human fact-checkers agreed with other fact-checkers on the accuracy of claims 64 percent of the time — an illustration of just how challenging the task can be.
Boosting performance
Sallami points to one reason for current shortcomings in accuracy: LLMs often struggle when they’re given ambiguous information. So they might flounder when they find contradictory evidence on the web, or misinterpret the evidence if the context for a claim isn’t clear. The statement “Mark Carney is prime minister,” for example, is true only in Canada. Instead of forcing her model to immediately produce an answer as to whether a claim is correct or not, she’s training it to recognize when a claim is ambiguous and to ask the user for more information.
Some developers explicitly instruct their LLMs to state when there is insufficient evidence to back up a claim, says Lademi Aborisade, an investigative journalist at the Nigerian nonprofit Center for Journalism Innovation and Development. In 2024, the organization launched the Dubawa fact-checking bot, an LLM-based tool people can message on WhatsApp that cross-checks claims with articles from reputable media sources. If there’s no available information, “the bot lets you know that there is insufficient evidence for the claim” rather than attempting a response, she says. In such cases, Aborisade and her colleagues can then thoroughly investigate claims and publish their findings online.
Other groups are using LLMs to analyze claims not for what is being said, but for how it is said. A project called AI4Trust, a collaboration funded by 15 European research institutions, developed such AI-based tools to fight disinformation — misinformation that is deliberately created and spread. Its platform includes video and audio analysis tools that look for signs of tampering or being AI-generated.
For one tool, experts prompted an LLM to sniff out 42 common characteristics of disinformation, such as alluding to a secret group of conspirators or using emotionally manipulative language. “We give very detailed instructions on what we want to detect in the text,” says Georgios Petasis of Demokritos, the National Center for Scientific Research in Greece, who led the project’s text content analysis. When compared with human fact-checkers, the LLM was in agreement 70 percent of the time. That’s enough to help journalists and fact-checkers flag suspicious claims that may be worth investigating further, says Petasis.
AI-generated misinformation, both intentionally damaging and simply false, is everywhere across social media. AI created this fake image of a hummingbird nesting inside a flower, something that scientists say would never happen in real life.
CREDIT: KNOWABLE MAGAZINE
Experts suspect that social media companies already use LLMs to detect misinformation. But the abundance of fake news on social media raises the question of whether the technology is effective or used extensively enough, or if companies are taking sufficient action against such content. While social media companies have taken a step back from moderating content on their platforms in recent years, West says, he suspects they may step it up again after recent court cases in California and New Mexico found Meta and Google liable for causing harm to young users. Though these court cases centered on the safety and addictive design of social media platforms, West reckons they may affect how companies address related problems like the amplification of misinformation on social media. (The issue of liability for AI-generated falsehoods, as delivered by AI overviews in search engines like Google, also reared its head in a June German court ruling.)
A statement from YouTube said the platform uses a combination of advanced detection systems and human reviewers to enforce its misinformation policies, which forbid certain types of misleading or deceptive content with serious risk of egregious harm, such as promoting harmful remedies or interfering with democratic processes. In the last quarter of 2025, the platform removed 11,337 videos for violating its misinformation policies. TikTok, Meta and X did not respond to requests for comment.
Beyond detection
LLMs can be helpful not only in identifying dis- and misinformation, but also in merely making sense of the vast hodgepodge of claims flitting about online. West and his colleagues, for instance, are using LLMs to track clusters of social media posts that spread misleading or false narratives and observe how those stories emerge, proliferate and evolve over time.
One US example is the “stop the steal” conspiracy theory that became widespread after Joe Biden was elected president in 2020; numerous posts contained false allegations of widespread voter fraud as well as real but misleading information, such as a video that showed poll watchers being denied access to a polling station but not their later admission once officials realized they had made an error.
Summarizing the larger narrative beneath those countless individual posts is challenging, but West finds that LLMs are fairly good at this kind of labeling task. Where crisis managers, journalists and fact-checkers don’t have the resources to tackle each individual claim, LLMs can help by quickly characterizing the big-picture narrative so it can be evaluated and, if needed, debunked. “If you can address the large-scale narrative of what’s going on,” West says, “then it helps you address a much bigger thing.”
Researchers have found that AI can not merely clarify, but also change, people’s misinformed beliefs. In one 2024 study in Science, 2,190 Americans who believed in a conspiracy theory such as the Moon landing being a hoax chatted with a version of ChatGPT that had been instructed to change the person’s mind. Remarkably, the chatbot reduced people’s belief in the theory on average by 20 percent, a higher success rate than other interventions like therapy targeting the underlying psychology that promotes conspiracy adherence.
(Science recently alerted readers to issues with the study’s data; the authors have submitted corrected data and report that the original results still stand.)
Computational social scientist Thomas Costello of Carnegie Mellon University, who co-led the study, says this result shows that fact-based arguments can work as long as time and effort is spent on conducting them, which is possible given the infinite patience of LLMs. “They’re actually incredibly good at using reason and evidence to talk someone out of a particular belief,” Costello says.
Even with AI’s growing ability to research and reason, however, experts generally don’t believe that this technology could — or should — be treated as a reliable substitute for professional human fact-checking, whether the goal is to dissuade, detect, debunk, label or restrict. Rather, most scientists see AI primarily as a way to sift through the ever-growing barrage of misinformation and flag content that journalists, fact-checkers or online platforms can further investigate. After all, blindly trusting the outputs of AI tools like LLMs — which are just as biased as the human-compiled data they were trained on — is one big part of what gets us into trouble with misinformation in the first place.
“We cannot just rely only on the AI,” says Nguyen, who likens training AI models to raising a child. “Of course we want the child to be autonomous, but we need to observe the behavior of the system and try to correct and guide it.”
(Knowable Magazine’s journalism is fact-checked by humans.)
