Mark Müller

New paper: Frontier Coding Agents Find Different Bugs Than Maintainers Fix

In proactive bug finding, agents report fewer than 20% of the bugs humans fixed. But they report a lot of other bugs, most of which are valid. This points towards an important distributional mismatch.

1/10
Went through Anthropic’s S-1. Two things stand out:

The unit economics look like normal software, at least for now
Over 90% of their own code is written by Claude, and the risk factors read like a list of what goes wrong at that scale 🧵

New benchmark publication with @logic_star_ai and @the_sri_lab:

Frontier coding agents struggle to generate strong test suites.

Love that NeurIPS is creating an incentive for reviewers to write high-quality reviews! Feels even more valuable with normal registration being sold out for months.

Having a blast demoing LogicStar at AI Launchpad as one of only 6 start-ups selected from over 100.

If you also want to learn how we catch issues before they turn into incidents, hit me up.