The release comes just days after a bug in AI-generated code cost Moonwell users nearly $2.7 million in crypto.
OpenAI and crypto venture capital firm Paradigm on Wednesday released a tool that evaluates AI agents’ ability to identify, patch, or exploit smart contract vulnerabilities.
The tool, EVMbench, draws from 120 vulnerabilities identified over 40 prior smart contract audits, as well as “vulnerability scenarios” drawn from audits of Paradigm’s forthcoming Tempo blockchain.
The release comes days after a bug in AI-generated code cost users of crypto protocol Moonwell nearly $2.7 million in crypto.
One Moonwell software engineer said the code in question had passed an audit from crypto security firm Halborn.
So-called agents are instances of artificial intelligence that can complete complex tasks in the digital world. They can write software, purchase theatre tickets, and conduct research on behalf of their users.
EVMbench data shows that OpenAI’s latest agentic coding model, GPT-5.3-Codex, more than doubled the effectiveness of an earlier model, GPT-5, in exploiting vulnerabilities in smart contract code. But its success in finding and fixing vulnerabilities “remain below full coverage,” OpenAI said in a news release.
“Agents perform best in the exploit setting, where the objective is explicit: continue iterating until funds are drained,” the company said.
“In contrast, performance is weaker on detect and patch tasks. In ‘detect’, agents sometimes stop after identifying a single issue rather than exhaustively auditing the codebase. In ‘patch’, maintaining full functionality while removing subtle vulnerabilities remains challenging.”
A model from Anthropic, Claude Opus 4.6, scored the highest mean result in detecting software vulnerabilities. GPT-5.3-Codex achieved the highest results in patching and exploiting smart contracts.
OpenAI cautioned that EVMbench doesn’t capture the true challenge of securing smart contracts, given the limited sample of vulnerabilities used to build the tool. And it can’t reliably determine whether agent-found vulnerabilities are, in fact, false positives.
Hacks have long bedevilled the crypto industry. Non-reversible transactions make crypto protocols’ smart contracts an attractive target for cybercriminals.
As of Wednesday evening, protocols suffered more than $108 million in hacks and exploits in 2026, according to DefiLlama data.
