SGLang

This account is verified. Learn more

@sgl_project

Run LLMs fast at any scale 🔗 github.com/sgl-project/sg…
Join our community slack.sglang.io
For AI tech blogs & deep-dives 👉 @lmsysorg

Palo Alto sglang.io Joined May 2025

58 Following 13K Followers 637 Posts

Latest posts 5

Pinned post

SGLang @sgl_project 3d

Kimi K3.1 Pro replaces dense MLA with sparse attention (DSA). It's much more efficient and easier to serve.

SGLang delivers on Day 0:
· Up to 33.7K tok/s per GPU (AgentX, GB300)
· 450 tok/s single user with DFlash2 + MegaKernel
· NVIDIA Vera Rubin / Blackwell / Hopper, AMD, TPU, Trainium, PPU

0:29

Kimi.ai @Kimi_Moonshot 3d

Releasing the model weights of Kimi K3.1 Pro: huggingface.co/moonshotai/Kim…
We'll keep advancing open intelligence.

View quoted post on X

SGLang @sgl_project 36m

SGLang Summit 2026 breakout sessions welcome teams shaping how AI learns, understands, and verifies.

Explore the talks below 🧵

📍 Nov 12–13 · Fort Mason, San Francisco
Open to Expo Pass holders. Get yours now: luma.com/sglang-summit

More talks to be announced

SGLang @sgl_project 1d

Breakout sessions at SGLang Summit 2026 cover SGLang on GPUs, CPUs and AI accelerators: new platforms, performance tuning and real workloads.

📍 Nov 12–13 · Fort Mason, San Francisco
Breakout access included with expo pass: luma.com/sglang-summit

Explore select sessions below 👇
More sessions to be announced

SGLang @sgl_project 2d

Thank you @radixark, @NVIDIAAI, @AIatAMD, @Kimi_Moonshot and @SemiAnalysis_ for this incredible run of day-0 model support and agentic workload optimization!

On GB300 NVL72, SGLang delivers 420+ tok/s per user at low concurrency and 31K+ tok/s per GPU at high concurrency. We are also excited to see B300, GB200 NVL72, B200 and MI355X curves available on day 0!

Kimi K3.1 Pro recipes are now available in the SGLang Cookbook: docs.sglang.io/cookbook/autor…

More optimizations and more hardware are coming to AgentX soon 🫡

docs.sglang.io Kimi-K3.1-Pro - SGLang Documentation Deploy Moonshot AI's Kimi K3.1 Pro with SGLang: 2.8T parameters (104B active) and a 1M context, on NVIDIA Vera Rubin, Blackwell and Hopper, AMD Instinct, PPU and Google TPU.

SemiAnalysis Verified account @SemiAnalysis_ 2d

Congrats to the SGLang team @sgl_project and @radixark on the amazing Day-0 Kimi K3.1 performance! Notably, the team shipped megakernel support out-of-the-box!

View quoted post on X

SGLang @sgl_project 2d

Congrats to the Google TPU team, and thanks @SemiAnalysis_ for a clear look at where TPU stands today and the potential it holds for production agentic workloads.

We’re excited to see @radixark work with Google to build a fully pipelined megakernel for Muse Spark Open, with deep optimizations for long-context agentic workloads. On TPUv7, the performance is now very competitive with popular GPU options.

More models on SGLang-TorchTPU are in the works, and TPU will only get stronger for anyone serving AI at scale 🧡

SemiAnalysis Verified account @SemiAnalysis_ 2d

The Iron(wood) Throne:
Google's TPU Stakes Its Claim on Inference
Viable Alternative of CUDA Moat,
NVIDIA Working Nights & Weekends,
Jensen’s Counter Submission, Kimi K3,
Meta Muse Spark Open, GLM5.3,
TorchTPU, SGLang, vLLM, llm-d, Ray
newsletter.semianalysis.com/p/tpu-strikes-…
View quoted post on X