Twitxy Saved

Philipp Moritz

Verified account This account is verified. Learn more

@pcmoritz

Co-founder and CTO at @anyscalecompute. Co-creator of @raydistributed. Interested in ML, AI, computing.
San Francisco Joined April 2017
8 Following 1.2K Followers 128 Posts

Latest posts 5

Philipp Moritz Verified account @pcmoritz
We just merged a clean Qwen 3.5 implementation for SkyRL's Jax backend: github.com/NovaSky-AI/Sky… Currently only for dense models, but should be easy to adapt to MoE models, contributions welcome! Also if anybody wants to contribute chunkwise training for the gated delta net or layer stacking for the model, it would be welcome!
github.com [tx] Implement Qwen 3.5 model architecture by pcmoritz · Pull Request #1228 · NovaSky-AI/SkyRL This PR implements the Qwen 3.5 model architecture, supporting mixed linear and full attention layers. For now we don't stack the layers yet to keep it simple. This PR also doesn't ...
1 replies 7 reposts 45 likes 4.2K views View on X
Philipp Moritz Verified account @pcmoritz
We just released the full SkyRL + Tinker integration: novasky-ai.notion.site/skyrl-tinker This is the evolution of what we have been working on with SkyRL tx and now also supports the SkyRL train backend (fsdp and megatron) which could previously only be used through Python APIs. I'm excited about the standardization the Tinker API will bring to the ecosystem and hopefully having a great open-source implementation will accelerate the adoption!
1 replies 9 reposts 44 likes 2.2K views View on X
Philipp Moritz Verified account @pcmoritz
We are excited to announce the release of SkyRL tx 0.3 novasky-ai.notion.site/skyrl-tx-v030, our MultiLoRA native inference and training engine that exposes the Tinker API. A lot has happened since the last release, in terms of big features we implemented expert parallelism, the DeepseekV3 model architecture (e.g. GLM 4.7 Flash) and a number of features to support longer context! Also, there are lots of smaller improvements and a few small bug fixes.
7 replies 7 reposts 68 likes 13K views View on X
Philipp Moritz Verified account @pcmoritz
Thanks a lot for the call-out @tinkerapi For anybody who is interested, we have made a bunch more releases since 0.1.0 came out, they are listed in github.com/NovaSky-AI/Sky… (and 0.3.0 with expert parallelism, a bunch of long context optimizations, and DeepSeekV3 is coming very soon).
Tinker Verified account @tinkerapi Feb 4
SkyRL-tx by @BerkeleySky is an open-source backend that implements the Tinker API itself, letting users train on their own hardware. It supports end-to-end RL, faster sampling, and gradient checkpointing — giving users flexibility and control.

novasky-ai.notion.site/skyrl-tx-v010
View quoted post on X