Twitxy Saved

Will Held

@WilliamBarrHeld

Open LLM Training @ marin.community

Formerly ML PhD w/ @Diyi_Yang, 🦙 @AIatMeta, Assistant @GoogleAI, اللغة العربية @NYUAbuDhabi
Burqueño
WilliamHeld.com Joined October 2012
1.2K Following 3.5K Followers 1.3K Posts

Latest posts 5

Pinned post

Will Held @WilliamBarrHeld
There’s an enormous amount of open training data on Hugging Face. What does it take to make it work together 🤗?

For Marin’s 535B run, we built on 25T tokens from 152 datasets with licenses permitting training.

Here’s the work between downloading those and training a model 🧵
23 replies 94 reposts 960 likes 126K views View on X
Will Held @WilliamBarrHeld
After great meetups for DSPy and Terminal Bench, was fun to bring together ~100 folks at Laude to talk about our progress on pretraining for the Marin hero run!

@LaudeInstitute is leading so much core community building needed for open work to meaningfully advance AI research.
Laude Institute @LaudeInstitute 2d
After finishing the largest-ever live pretraining run, @MarinCommunity released checkpoints on @huggingface yesterday — great timing for a community meetup at Laude Lab!

@WilliamBarrHeld's ask for the audience: “Find what Marin is doing wrong and send better ideas back.”

Proud to support Marin and Atlantic-CC as Slingshots. Thanks to the Open Athena team for helping us get everyone together! @percyliang @dlwh @andykonwinski
View quoted post on X
1 replies 2 reposts 24 likes 1.1K views View on X
Will Held @WilliamBarrHeld
Figuring out principled AI equivalents of wind tunnels, model organisms, tabletop experiments (etc.) is critical for academia to keep advancing our understanding of AI in an era of massive compute.

Very cool work from @suze_va, @LukeBailey181, @rckpudi et al. using Delphi!
Suze van Adrichem Verified account @suze_va Nov 3
Can we forecast benchmark accuracy from models that score zero?

We (co-leads @LukeBailey181 @rckpudi) turn a benchmark into a series of easier hinted versions. Jointly modeling scale C and hint level H enables forecasting GSM8K acc (H=0) within 3 pts from 5600x less compute 🧵
View quoted post on X
0 replies 4 reposts 28 likes 1.3K views View on X
Will Held @WilliamBarrHeld
.@N8Programs was ahead of their time with this tweet

(w.r.t. rikiyat.github.io/blog/weight-sp… going mega-viral today)
rikiyat.github.io Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights We find that frontier LLMs are surprisingly good at natively reading and writing neural network weights much like code, sometimes with tools such as a Python kernel and sometimes from the raw weights...
N8 Programs ⏹️🤝⬆️ Verified account @N8Programs Feb 24
Beat it by having Codex hand-craft weights:
gist.github.com/N8python/02e41…

100% accuracy on 10 million random test cases w/ only 343 parameters. As a bonus, it uses the vanilla Qwen3 architecture, just with the right weights.
View quoted post on X
0 replies 2 reposts 13 likes 742 views View on X
Will Held @WilliamBarrHeld
Re: old "wordcel" vs. "shape rotator" discourse in research.

The expectation was that people who were differentiated by communication skills would lose their edge to AI.

The opposite seems true right now. Those who make arcane research clear are more valuable than ever!