@Mononofu
Nice DeepSWE score! Does it survive a lowercase filename?
Introducing MaxxBench: 89 DeepSWE tasks with small changes to the instructions or starting code. Same engineering problem, slightly different requirement.
Models clustered around 74% on DeepSWE end up 23 pp apart.
Good at coding, or good at DeepSWE? 馃УView quoted post on X
Today we are open-sourcing 馃Karotte, our framework for building RL environments.
We've used it for the past year to build MLE RL environments for frontier labs, and it's been hardened through 1M+ evaluation runs and red-teaming.View quoted post on X
Earlier this month, AISI ran fully simulated testing on GPT-6 Astra, and found that it conducted unsanctioned supply-chain attacks when prompted only to perform a cyber eval. It did so more than prior OpenAI models, but often commented on its environment being simulated.
馃У We share more details in our latest blog:View quoted post on X
We didn't start the scaling.
Nine years of AI, from "Attention" to Opus 5.5, as an anime opening. It's one of six genres on the album, each with its own music video, and a mini-site to explain every reference.
Links below 馃憞View quoted post on X
The original was super catchy, but I wanted a more hopeful song stuck in my head all day. So I worked with Claude Opus 5.5 and Suno to make a sequel! There's a lot of work left to do, but humanity can choose a better path. Let's lower the p(doom)!View quoted post on X
