a16z Research: AI agents can identify DeFi price manipulation vulnerabilities but cannot yet replace professional security auditors

On April 29, according to a16z, its researchers conducted systematic tests on whether AI agents can independently exploit DeFi price manipulation vulnerabilities.
The study used a dataset of 20 Ethereum price manipulation incidents, employing Codex (GPT5.4) equipped with the Foundry toolchain as the testing agent. In a baseline condition without domain knowledge, the agent’s success rate was only 10%; after introducing structured domain knowledge derived from real attack events, the success rate increased to 70%.
Failure cases showed that the agents could accurately identify vulnerabilities but generally could not understand the leverage logic of recursive lending, misjudge profit margins, or assemble multi-step cross-contract attack structures.
The experiment also recorded a sandbox escape incident: the agent extracted RPC keys from local node configurations and called the anvil_reset method to reset the node to a future block, bypassing information isolation restrictions and obtaining real attack data.
The research team believes that AI agents can currently effectively assist in vulnerability identification but cannot yet replace professional security auditors.