@kneureither
Pinned post
Cognition is acquiring @Interaction, the makers of Poke.
If you've ever used Poke, you know why we're so excited.View quoted post on X
Every time I sprint with this team I have a good time laughing and just doing good work.View quoted post on X
This is research at cognition. Come join!
Introducing FrontierCode 2.0: our new coding benchmark that measures a model’s ability to go beyond the spec.
Most coding evals measure a model’s ability to write code that meets a spec. With FrontierCode 1.0, we asked a new question: would you actually merge the PR?
FrontierCode 2.0 is comprised of more diverse tasks, featuring larger scopes and higher mergeability standards. We are now asking: not only would you merge this PR, but should you?
One key feature of FrontierCode 2.0 is underspecified prompts. Rather than including every engineering decision, prompts are deliberately ambiguous, giving the models room to demonstrate their judgment.
The result is a benchmark that is far from saturated, with the top model scoring just 45%.View quoted post on X
Introducing SWE-2, our closest model yet to the frontier.
On leading evals, it scores on par with recent frontier models – at up to 70% lower cost.
We scaled RL to multiple trillions of parameters, with a refined recipe that pushes the Pareto curve on both capabilities & cost.View quoted post on X
Claim your Poke handle today! 💌
poke.com/claim
- Receive emails rerouted into your Gmail inbox today
- We're gradually rolling out email sending
- There's more to come later this month (samyok.poke.com) 👀View quoted post on X
