You should never trust an AI chatbot with your deepest, darkest secrets. A Chinese law enforcement official learned this the hard way when they decided to use ChatGPT as their personal diary.
A major influence campaign was uncovered not by the CIA or MI6, but by OpenAI, the company behind ChatGPT, all because a user was treating the chatbot like a private notebook to plan his next ‘smear’ operation.
According to a new report by OpenAI, a Chinese law enforcement official unknowingly revealed details of an operation aimed at intimidating Chinese dissidents living abroad, including by impersonating US immigration officials.
OpenAI said the user appeared to rely on ChatGPT as a running journal diary to document the alleged covert campaign of suppression.
In one instance, operators allegedly warned a Chinese dissident living in the US that their public statements had violated American law. In another case, forged documents purportedly from a US county court were made to pressure social media platforms into removing a dissident’s account.
The campaign, according to OpenAI, involved hundreds of participants and thousands of fake online accounts operating across multiple social media platforms.
“This is what Chinese modern transnational repression looks like,” Ben Nimmo, principal investigator at OpenAI, said ahead of the report’s release, as per CNN. “It’s not just digital. It’s not just about trolling. It’s industrialised. It’s about trying to hit critics of the CCP with everything, everywhere, all at once.”
ChatGPT itself was not used to generate most of the campaign content. Instead, OpenAI said the tool functioned mainly as a planning and record-keeping space, while other systems produced and distributed material online.
The company banned the account after identifying suspicious activity.
As part of the campaign, a Chinese official allegedly attempted to fabricate the death of a Chinese dissident.
The ChatGPT user described creating a false obituary along with images of a gravestone, which were then circulated online. OpenAI investigators later matched these claims with real rumours that surfaced in 2023 about the dissident’s supposed death, reported at the time by Voice of America’s Chinese-language service.
Beyond Chinese dissidents, the campaign also extended to world leaders.
According to OpenAI, the user asked ChatGPT to draft a plan to undermine incoming Japanese Prime Minister Sanae Takaichi by amplifying anger over US tariffs on Japanese goods on social media.
As the era of traditional cyber warfare evolves, AI is emerging as the next major front in warfare. The US and China are no longer just competing on code and language models; they are racing to weaponise the Artificial Intelligence itself.