Pinned post
Sumanth Hegde
Verified account This account is verified. Learn more@sumanthrh
Post-training @anyscalecompute. Prev - @UCSanDiego, @C3_AI, @iitmadras. Machine Learning and Systems. Intensity is all you need.
27 Following 1.2K Followers 477 Posts
Latest posts 5
Avi + Agents + GPUs = Profit
Super excited to share some of the work I’ve been doing on supporting 1M token context training on SkyRL for GLM 5.3 Flash, 64x higher than the out of the box Megatron baseline! 🚀
This work was done in collaboration with the amazing folks at @LilaSciences! 🤩
Getting there took key algorithmic and systems optimizations in four main areas:
1) DeepSeek Sparse Attention: run DSA on a sparse kernel; chunk and shard its quadratic indexer.
2) Activation recompute and offloading: recompute activations in backward and offload saved inputs to CPU.
3) Context parallelism: split each sequence across GPUs, with new CP support for KDA and a fix for DSA.
4) Avoiding unnecessary precision upcasts: keep FP32 copies out of memory and off the network.
For more details on our implementations and our results, check out our full technical blog here: blog.skyrl.ai/blog/1m-contex…!
This work benefited from the contributions of many people along the way. Big thanks to @pcmoritz, @erictang000, @sumanthrh, @CyrusHakha, @richliaw from @anyscalecompute and Keith Truongcao, Tushar Khot, Vivek Subramanian, Semih Kara from Lila Sciences for all the help and guidance!View quoted post on X
Great to see this out!
Datadog's new SOTA model for investigating production incidents. Their in-house training stack builds on top of SkyRL! 🚀🚀🚀
Datadog's new SOTA model for investigating production incidents. Their in-house training stack builds on top of SkyRL! 🚀🚀🚀
Meet Scout 1.0, our first AI model built for investigating production incidents.
Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier.
We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence.
Learn more and sign up for the preview: bit.ly/3TJknGOView quoted post on X
Check out this blog series from Coinbase on how they trained and deployed their own fraud detection agent on Anyscale!
Great to see more cases of SkyRL in production - enabling a 9B model to punch above its weight!
Great to see more cases of SkyRL in production - enabling a 9B model to punch above its weight!
People who are really serious about applied AI should train their own models.View quoted post on X
Sharing our “Owning Intelligence” series on how we built, evaluated, and post-trained a fraud agent at Coinbase—from using frontier models to owning a model optimized for our task.
Part 1 — Build
lnkd.in/grtg_gfF
Part 2 — Evaluate
lnkd.in/g-6DKdZ8
Part 3 — Own
lnkd.in/g9ET_4at
@akshit_trehan
@anyscalecompute, @erictang000 , @sumanthrh , @seiji_________ ,
LFG
Read more about our experiment using SkyRL here: afterquery.com/blog/improving…View quoted post on X
To learn more about our tasks and reward design, reach out to us at research@afterquery.com
