Genians confirmed hackers bypassed ChatGPT’s ID restrictions using altered prompts.
A North Korea-backed hacking group called Kimsuky used ChatGPT to build a fake South Korean military ID and launched a phishing attack targeting journalists, researchers, and human rights workers, according to cybersecurity firm Genians.
The email carrying the deepfake ID contained malware built to steal information from the recipients’ devices. This campaign is part of a wider pattern of North Korean cyber-operations using AI to carry out global espionage.
The phishing email was disguised to look like it came from a real military account, ending in .mil.kr. There was no photo attachment, no image of the ID. Instead, there was a hidden payload ready to infect the target’s system.
Genians confirmed that the fake military ID was generated using ChatGPT after bypassing the platform’s restrictions. When asked to generate the ID directly, the tool initially refused. But researchers managed to bypass the block by changing how the prompt was written.
Once the prompt was rewritten, the system produced a convincing draft, enough to bait victims into clicking the embedded malware.
The same strategy wasn’t limited to South Korea. In August, the AI firm Anthropic said it found North Korean hackers using its Claude Code model to apply for remote jobs with U.S. Fortune 500 companies.
The hackers used Claude to pass coding interviews, create full work histories, and even do technical assignments after getting hired. The operation gave North Korea direct access to corporate systems inside the U.S. without needing to break through any firewalls.
In February, OpenAI banned accounts tied to North Korea that had used its tools to make fake résumés, cover letters, and social media posts. These profiles were designed to trick people into helping the regime’s campaigns, knowingly or not.
Mun Chong-hyun, director at Genians, said these new techniques show how North Korea has now integrated AI at every stage of the hacking process, from planning and tool creation to phishing and impersonation.
“Attackers can use AI to map scenarios, write malware, and even pretend to be recruiters,” Mun said. The U.S. government has said that North Korea’s cyber efforts are part of a bigger operation.
They believe the regime in Pyongyang is using hacking, crypto theft, and shadow IT contracts to collect data, gather intelligence, and generate funds to support its nuclear weapons program while dodging international sanctions.
Back in 2020, the U.S. Department of Homeland Security issued a formal advisory describing Kimsuky as being “most likely tasked by the North Korean regime with a global intelligence-gathering mission.”
The group has been active since 2012 and has focused its attacks on foreign policy experts, think tanks, and government agencies in South Korea, Japan, and the United States.
Most of the time, they use spearphishing emails to gain entry into systems, extract sensitive information, and track high-level discussions about nuclear strategy, sanctions, and regional security.
The Genians report also confirmed that the latest victims were carefully chosen. The hackers went after people with strong ties to issues surrounding North Korea, such as activists, journalists, and defense researchers. It’s still unknown how many devices were actually compromised.
But the fact that they were able to spoof a South Korean military email domain and insert malware into a seemingly harmless message shows how dangerous this method is.
During the investigation, Genians tried replicating the hackers’ method using ChatGPT themselves. Their experiment confirmed that while ChatGPT is designed to block illegal content like fake government IDs, attackers were still able to work around it with minor changes in language.
The end result was an ID template that didn’t look suspicious until it was too late.
CISA, FBI, and CNMF have now called on anyone working in sensitive fields related to North Korea to tighten their security. They warned that Kimsuky continues to use phishing, fake recruiter accounts, and spoofed domains to get into networks.
Their main suggestions include enabling multi-factor authentication, rolling out phishing awareness training, and setting up stronger filters for suspicious emails.
The U.S. intelligence community has long said that cyber operations are now one of North Korea’s primary tools for bypassing sanctions.
Don’t just read crypto news. Understand it. Subscribe to our newsletter. It’s free.
