Pinned post
Up to 5× faster RL training steps, on the same GPU budgets.
Our internal auto-tuner optimizes the full RL loop: training, inference, and environment execution. It profiles each job, forecasts promising configuration changes, tests them in real runs, and follows the bottleneck as it moves.
Across 210 runs on NVIDIA B300 GPUs, we tested four dense and MoE models on math and agentic coding workloads. All eight jobs improved: 1.42-4.93x faster steps, with a median speed-up of about 3.5x.
One revealing result: nearly doubling inference-engine throughput in a benchmark didn’t shorten the RL step. Sandbox execution absorbed the gain.
That’s why we measure the complete job, alongside numerical checks and longer reward comparisons. GPU counts stayed fixed; some configurations added sandbox hosts.
We’ve shared the system, results, and five engineering lessons, including what didn’t work, in the article below:
Our internal auto-tuner optimizes the full RL loop: training, inference, and environment execution. It profiles each job, forecasts promising configuration changes, tests them in real runs, and follows the bottleneck as it moves.
Across 210 runs on NVIDIA B300 GPUs, we tested four dense and MoE models on math and agentic coding workloads. All eight jobs improved: 1.42-4.93x faster steps, with a median speed-up of about 3.5x.
One revealing result: nearly doubling inference-engine throughput in a benchmark didn’t shorten the RL step. Sandbox execution absorbed the gain.
That’s why we measure the complete job, alongside numerical checks and longer reward comparisons. GPU counts stayed fixed; some configurations added sandbox hosts.
We’ve shared the system, results, and five engineering lessons, including what didn’t work, in the article below:
0:20
Inside Nebius’s RL auto-tuner: how optimizing training, inference and sandboxes together delivered up to 5× faster steps on fixed GPU budgets.View quoted post on X
And five surprising lessons from 210 runs, including why faster inference can leave RL just as slow. x.com/i/article/2117…
