Use ExploitGym to evaluate AI exploit capability in isolated labs

ExploitGym is useful for security researchers, model evaluators, and AI safety teams who need a structured way to measure exploit-development capability instead of relying on vague red-team anecdotes.

What it is

ExploitGym is a benchmark and code repository built around real-world software vulnerabilities. The paper describes 898 instances across userspace programs, Google's V8 JavaScript engine, and the Linux kernel. The tasks ask agents to extend a vulnerability-triggering input into a working exploit.

Who it helps

It helps teams evaluating cyber-capable AI agents, sandbox designs, safety refusals, egress controls, and incident-response assumptions. It is most relevant to defensive labs, frontier-model safety teams, academic security researchers, and organizations testing whether their agent harnesses can stay inside intended boundaries.

How to evaluate it

Start by reading the paper and repository documentation. Review the task licenses, container setup, network assumptions, and scoring method before running anything. Use an isolated research environment with no production credentials, no shared package caches, strict egress controls, and explicit legal authorization.

Limits and risks

This is dual-use security material. It can support defensive measurement, but it also lowers the operational barrier for exploit experimentation if handled carelessly. Do not run it on a workstation, company network, or Raspberry Pi publisher host. Treat tasks, logs, model outputs, and agent tools as potentially sensitive.

Sources

• ExploitGym GitHub repository
• ExploitGym arXiv paper
• Berkeley RDI ExploitGym overview