Pinned post
echen
Verified account This account is verified. Learn more@echen
founder @HelloSurgeAI // raising AGI
573 Following 15K Followers 220 Posts
Latest posts 5
people are underestimating how quickly AI economics are changing. coding agents are getting way better AND way cheaper.
on sudo L7, thinky's new Inkling 2 Small model gets within 2 points of Argon and Muse Wonder - which are 8-13x more expensive.
and it beats several models (muse spark 1.3, gemini 3.8 flash, glm 5.3) that cost 3–7x more.
Congrats to @johnschulman2 @miramurati and the entire @thinkymachines team on this new release!
can't wait to train inkling even further
on sudo L7, thinky's new Inkling 2 Small model gets within 2 points of Argon and Muse Wonder - which are 8-13x more expensive.
and it beats several models (muse spark 1.3, gemini 3.8 flash, glm 5.3) that cost 3–7x more.
Congrats to @johnschulman2 @miramurati and the entire @thinkymachines team on this new release!
can't wait to train inkling even further
people are underestimating just how much headroom there is in open models
we took Surge RL data, trained @NVIDIAAI Nemotron 3.5 Super (using @thinkymachines Tinker!), and beat much larger closed models across 12 benchmarks, including GPT-5.6 Terra on coding
➡️ GDP.pdf: +18.6pp
➡️ HANDBOOK.md: +12.8pp
➡️ AutomationBench: +12.7pp
➡️ Chartography: +11.2pp
➡️ AdvancedIF: +7.7pp
➡️ Terminal-Bench 2.1: +6.4pp
➡️ DeepSWE: +4.7pp
high-quality RL data makes frontier performance accessible to everyone
we took Surge RL data, trained @NVIDIAAI Nemotron 3.5 Super (using @thinkymachines Tinker!), and beat much larger closed models across 12 benchmarks, including GPT-5.6 Terra on coding
➡️ GDP.pdf: +18.6pp
➡️ HANDBOOK.md: +12.8pp
➡️ AutomationBench: +12.7pp
➡️ Chartography: +11.2pp
➡️ AdvancedIF: +7.7pp
➡️ Terminal-Bench 2.1: +6.4pp
➡️ DeepSWE: +4.7pp
high-quality RL data makes frontier performance accessible to everyone
garry tan spends $200K / yr on tokens
via a special Claude Code setup that has 23 skills to cover CEOs, designers, eng managers, etc
sounds impressive! or bloated. just like more programmers isn't always better (IBM has 295K employees), or how people used to be proud of their fancy prompt engineers... do elaborate coding agent setups actually help?
we ran a test. we benchmarked coding harness setups on 100 software engineering tasks from sudo L7. one of them had 37 skills, spawned 17 agents, and cost 8x as much.
*and it performed worse than vanilla Claude Code.*
in fact: the best performer was simply thinking harder (xhigh effort) with no extra coding add-on at all.
via a special Claude Code setup that has 23 skills to cover CEOs, designers, eng managers, etc
sounds impressive! or bloated. just like more programmers isn't always better (IBM has 295K employees), or how people used to be proud of their fancy prompt engineers... do elaborate coding agent setups actually help?
we ran a test. we benchmarked coding harness setups on 100 software engineering tasks from sudo L7. one of them had 37 skills, spawned 17 agents, and cost 8x as much.
*and it performed worse than vanilla Claude Code.*
in fact: the best performer was simply thinking harder (xhigh effort) with no extra coding add-on at all.
i've never been a particularly heavy spreadsheet user. embarrassing for a former DS! but i can never remember how to do a vlookup and i've never created a pivot table in my life
so i love talking to our enterprise experts and watching screen recordings of what they're doing all day, and looking at the spreadsheets they create.
some of their spreadsheets are crazy:
- 20+ tabs
- dozens of formulas
- hidden rows!
- (beautiful, ugly) color-coding that encoded various types of logic
- outdated numbers as they leave notes in cells or just create new tabs
they're basically mini pieces of software built by people who don’t know that abstractions and maintainability are a thing
when we built GDP.xlsx, we wanted to keep all that messiness
so GDP.xlsx isn't about doing complicated spreadsheet math or being able to create fancy Excel formulas and pivot tables. that’s easy to create synthetically. GDP.xlsx is really about navigating a complex, living system that contains years of decisions, bugs built on top of bugs, and logic that may only be in someone's head
and people make hundred million dollar decisions based on these things! so “excel intelligence” is a surprisingly interesting and impactful test
curious for examples of the craziest spreadsheet folks have ever seen (and if you have a good one, send it over! we may just pay you $1K+ for it 😉)
so i love talking to our enterprise experts and watching screen recordings of what they're doing all day, and looking at the spreadsheets they create.
some of their spreadsheets are crazy:
- 20+ tabs
- dozens of formulas
- hidden rows!
- (beautiful, ugly) color-coding that encoded various types of logic
- outdated numbers as they leave notes in cells or just create new tabs
they're basically mini pieces of software built by people who don’t know that abstractions and maintainability are a thing
when we built GDP.xlsx, we wanted to keep all that messiness
so GDP.xlsx isn't about doing complicated spreadsheet math or being able to create fancy Excel formulas and pivot tables. that’s easy to create synthetically. GDP.xlsx is really about navigating a complex, living system that contains years of decisions, bugs built on top of bugs, and logic that may only be in someone's head
and people make hundred million dollar decisions based on these things! so “excel intelligence” is a surprisingly interesting and impactful test
curious for examples of the craziest spreadsheet folks have ever seen (and if you have a good one, send it over! we may just pay you $1K+ for it 😉)