Twitxy Saved

Stanford AI Lab

Verified account This account is verified. Learn more

@StanfordAILab

The Stanford Artificial Intelligence Laboratory (SAIL), a leading #AI lab since 1963. ⛵️🤖 Emmy-winning video: youtube.com/watch?v=Cn6nmW…
Stanford, CA ai.stanford.edu Joined November 2018
330 Following 281K Followers 3.7K Posts

Latest posts 5

Stanford AI Lab Verified account @StanfordAILab
How do you teach an LLM a new document without it forgetting what it already knows? 📚 Check out Speculative Self-Distillation, a #NeurIPS2026 paper from @talaei_shayan, @chewing_a_gum, @ckodser and @Azaliamirh's group. The student writes, and the teacher steps in only where it matters.
Shayan Talaei Verified account @talaei_shayan 5d
Introducing Speculative Self-Distillation (SSD, NeurIPS 2026), which pushes the accuracy–efficiency–forgetting frontier of self-distillation.

In SSD, student and teacher (the same model with additional information) write the rollouts together. Like a private tutor, the teacher lets the student write and takes over only when the student drifts too far. As a result, SSD trains on rollouts that stay as close to the student's own writing (on-policy) as possible while the teacher keeps them on track.

📉 38–69% fewer training tokens than on-policy, matching its accuracy
⏱️ Reaches on-policy accuracy in ~130 vs ~414 min (Science Q&A)
🧠 Forgets less in a stress test: −1.4 vs −4.5 points for on-policy
View quoted post on X
14 replies 18 reposts 178 likes 12K views View on X
Stanford AI Lab Verified account @StanfordAILab
@StanfordAILab’s @talaei_shayan shipping incredible work! 🚀
Shayan Talaei Verified account @talaei_shayan Nov 4
Really happy to finally share Ripple-65B Preview! We trained it from scratch with a core team of five researchers, and it sets a new U.S. open-weight SOTA on Terminal-Bench 2.1 for agentic coding.

Three choices that shaped how we built it, and a few things I learned along the way.

1. We deliberately chose a dense model.

Many leading open-weight models are MoEs. For our use case, we wanted strong reasoning and low latency for agents going through long sequences of model and tool calls. Going dense also gave us a simpler training and serving stack and avoided expert-routing communication. That combination mattered a lot to us.

2. We bought no posttraining data.

A lot of the work went into building our own data generation and filtering pipelines, and making sure we covered the tasks we cared about. We built on open-source training and inference stacks throughout. I’m particularly happy about how much we were able to do as a small team by building on the community’s work.

3. We worked on the RL system and curriculum together.

Mixed RL meant dealing with very different rollout lengths, tool-call patterns and shares of the batch across domains. We tuned oversampling per domain and per problem to keep batch latency down, while managing policy staleness and trainer-inference mismatch. Score centering worked best among the corrections we tried; we used it alongside truncated importance sampling.

We also kept adjusting task selection and introducing harder problems as the model improved. Getting batches ready faster and keeping updates stable had to go hand in hand with giving the model useful problems to learn from.

Loved building this with @DZhang50, @esonghori2, @ethanboneh and @amiwilliamhuang. Huge thanks to our cofounders @annadgoldie and @Azaliamirh, and the whole @RicursiveAI family for making it possible. Really proud of what we built together.
View quoted post on X
4 replies 0 reposts 21 likes 9.4K views View on X
Stanford AI Lab Verified account @StanfordAILab
Check out Joules to GDP, which makes it possible for us to measure the economic value per unit of energy spent on compute!
Jon Saad-Falcon Verified account @JonSaadFalcon Nov 4
AI is using more and more of the world’s energy. But how much economic value are we getting from AI?

We’re introducing Joules-to-GDP (J2G), a framework that measures how well AI turns energy into economic value and shows which tasks, models, and hardware give the best return.

Across 44 occupations in finance, engineering, law, administrative work, and more in @OpenAI's GDPval, we arrive at three findings:

1) The tasks with the highest economic value are not the tasks with the highest economic value per joule. For example, CAD design earns ~10% more per task than building websites, but the latter earns ~27% more value per joule.

2) The most powerful model isn’t always the best use of energy. Gemma 4 31B delivers 56% of GLM 5.2’s net value with 23% of the energy, for 2.5x the value per joule.

3) More chips don’t guarantee proportionate returns. Moving the same model from 1 to 8 NVIDIA H100s uses 3.2x the energy but cuts completion time by only 22%, with about the same value.

With @SnorkelAI, we’re also releasing GDPWorks, a benchmark for economically valuable tasks across 36 additional occupations.

Joint work co-led by @amirzeinaalii and @Avanika15, with @Mersad_Abbasi, @Jon_Hartley_, John Hennessy, @HazyResearch, and @Azaliamirh.

More details below 🧵
View quoted post on X