Twitxy Saved

Sandeep Srinivasa

Verified account This account is verified. Learn more

@sandeepssrin

github.com/sandys/ Joined March 2009
1.7K Following 9.8K Followers 15K Posts

Latest posts 5

Pinned post

Sandeep Srinivasa Verified account @sandeepssrin
"Treat rollout as a service"

This is the coolest paper i have seen. And kind of vindication - I have been working on building browser based rollouts for better codegen rewards for many months now (w8-rl).

My thesis was always - that the big untapped piece in RL is infrastructure. Rollouts should be treated similar to how u treat Kubernetes pod scheduling.

But most data scientists are not devops engineers. And they smush rollouts inside the RL algo.
Robert Youssef Verified account @rryssf Mar 26
BREAKING: NVIDIA just proved that the AI agent training bottleneck everyone blamed on model capability was actually an infrastructure design error.

Every framework SkyRL, VeRL-Tool, Agent Lightning, rLLM, GEM embeds rollout inside the training loop. I/O-intensive execution fighting GPU-intensive optimization. Nobody separated them.

> Treat rollout as a service. Qwen 8B nearly doubles on SWE-Bench. The compute was always there. It was just fighting itself.

> Training an AI agent with reinforcement learning requires two fundamentally different workloads running simultaneously. Rollout is I/O-intensive spinning up sandboxed environments, executing tool calls, waiting on shell commands, scoring outcomes. Training is GPU-intensive forward passes, backward passes, gradient synchronization. Every existing framework runs both inside the same process. The result is constant resource contention: rollout workers block on disk I/O while GPU compute sits idle, and gradient updates stall while environments finish executing.

> NVIDIA audited every major agentic RL framework and found the same architectural decision in all of them. SkyRL keeps rollout control inside the training driver. Agent Lightning embeds rollout workers as child processes of the trainer if training stops, rollout stops. VeRL-Tool, rLLM, and GEM all keep environment management and trajectory collection inside the training stack. Not because it's the right design. Because it was easier to build that way and nobody had fixed it yet.

> ProRL Agent separates them completely. The rollout system runs as a standalone HTTP service. The trainer sends a task instance. The rollout server handles environment initialization, multi-turn agent execution, tool calls, reward computation, and returns a completed trajectory. The trainer never touches the execution environment. The two systems communicate through one interface and run on separate machines optimized for their respective workloads.

> The results are not subtle. Qwen 8B: 9.6% on SWE-Bench Verified under standard training. ProRL Agent: 18.0%. That's close to 2x on the benchmark the entire software engineering AI field uses to measure progress. Qwen 14B goes from 15.4% to 23.6%. Qwen 4B goes from 14.8% to 21.2%. Same models. Same data. Different infrastructure.

→ Qwen 8B on SWE-Bench Verified: 9.6% baseline → 18.0% with ProRL Agent
→ Qwen 14B: 15.4% → 23.6%
→ Qwen 4B: 14.8% → 21.2%
→ Throughput scales near-linearly with compute nodes added
→ Efficient bash optimization alone: shell command latency drops from 0.78s to 0.42s per action
→ GPU utilization with full system: 78% vs 42% without load balancing
→ Every existing framework audited: zero had decoupled rollout from training

The engineering insight that gets buried in the results: the problem compounds at every tool call. A typical software engineering rollout spans dozens of sequential environment interactions. Each one blocks. Each one accumulates latency. At scale with hundreds of parallel rollouts, tool execution becomes the dominant bottleneck not model inference, not gradient computation. The entire field was measuring model capability while the infrastructure was quietly eating half the compute budget.
View quoted post on X
Sandeep Srinivasa Verified account @sandeepssrin
Oooh
Musk goes anti-SGLANG
Elon Musk Verified account @elonmusk Nov 10
We have dumped JAX/SGLang and rewritten the entire RL and inference stack from scratch in C/C++, which took some extra time.

Training (aka pre-training) is done. Lot of cleanup was needed on the SFT data to get that mostly right. Now, 4.8 is in RL and the evals are starting to climb. We will do a soft rollout to Bot users later this week and see how it goes.

Grok 4.8 will bear no resemblance to 4.7 (thankfully). It is not yet incredible, but, this time, the foundations are built right, so I expect it to evolve to be something special.
View quoted post on X
Sandeep Srinivasa Verified account @sandeepssrin
The problem with this story is that its not a complete retelling.
Mainframes didnt evolve to PCs and end there. They went back to AWS cloud - which is as mainframeish as u can get.
And both the fruit company & the furniture company failed to capitalise on the fact they owned the OS/hardware. A grocery co won out.
Vivek Wadhwa Verified account @wadhwa Nov 6
My advice to India, as I wrote in @htTweets , is to stay away from this insanity and let Silicon Valley’s bets turn to toast. Stay out of the toaster. hindustantimes.com/opinion/todays…
View quoted post on X
Sandeep Srinivasa Verified account @sandeepssrin
And this is why you should own your intelligence, even if you're an applied AI startup. And think the labs are friendly to you.
Anthony Pompliano 🌪 Verified account @APompliano Nov 5
Let me tell you a story about how unethical Anthropic is.

In April, the Anthropic team began heavily courting our engineering team to tell them corporate secrets, technical details, and future plans for @cfosilvia.

They pretended their interest was due to our exploding popularity and accelerating compute usage, but it quickly became apparent that Anthropic was fishing for details on how we built our product and why it was so popular.

This included the Anthropic team trying to wine-and-dine our engineers, including a infamous steak dinner where someone told our team "don't worry about it, this is on Dario."

Our team was not born yesterday, so we quickly realized what was happening and refused to share sensitive information and technical details with the Anthropic team.

As we suspected, Anthropic was trying to get information from us, so they could build a competing product. Some people on their team must have a guilty conscious because they messaged us a few days ago to give us "a heads-up" that they were launching a competing product.

Thankfully, the Silvia engineering team has been aggressively working to get off the frontier models for the last ~ 6 months. That effort has been highly successful, because our compute costs are ~ 97% lower when we use our own model & hardware, the latency is lower, and the accuracy of answers materially increases as well.

The sad thing for Anthropic is they have now released an inferior product and they had to sell their soul to do it.

The only thing worse than being a trillion-dollar company that can't beat a 10-person team is being a trillion-dollar company that no one trusts and everyone knows is unethical.

Serious investors continue to choose @cfosilvia over ChatGPT, Claude, and other frontier companies because these products are not optimized to solve the actual user problem.

Today, Anthropic launched a Claude wrapper that is masquerading as a personal finance product. Best of luck to them.

But I will take the Silvia misfits vs the Anthropic losers any day of the week.

if you want to support people trying to do things the right way, and help investors be more informed, then consider signing up for Silvia today: silvia.com
View quoted post on X