FixBench

An open model fixing real curl security bugs, with curl's own test suite as the judge.

Every instance is a published curl CVE whose fix commit added a regression test. The agent gets the public advisory and the vulnerable checkout. It reads the code, edits it, and builds curl and runs the tests itself in a network-less sandbox, then iterates. It never sees the maintainers' fix. A CVE counts as fixed only if a fresh build of the final patch passes the regression test and every nearby test that passed before. Every model call runs on Nebius Token Factory.