Twitxy Saved

Yi Pan

Verified account This account is verified. Learn more

@conlesspan

PhD Student @BerkeleySky | Systems & AI | conless.dev
Prev. BS @sjtu1896, RA @UWSyFI
Joined August 2019
358 Following 554 Followers 115 Posts

Latest posts 5

Yi Pan Verified account @conlesspan
Congrats to @XYLi_Bruce and the team on QDA! It turns PTQ recipe design into a real closed-loop search and beats the best public recipes on multiple settings ๐Ÿš€
Xingyang Li Verified account @XYLi_Bruce 1d
Agent loops need fast feedback, and LLM quantization doesn't have it: evaluations of reasoning models are slow, and PTQ methods are hard to port and combine.

Meet QDA. A coding agent designs the quantization recipe for each model and precision, with a KL score that takes minutes and a library of 20+ stackable PTQ methods.

๐Ÿ“ˆ Beats the best public recipe on 11 of 12 model-precision pairs: +0.9 / +1.4 / +2.2 pts at W4A16 / W4A8 / W3A16
๐Ÿ”ฅ GLM-5.3-Flash (320B) at NVFP4: 98.3% of FP8 accuracy, vs 96.9% for NVIDIA's release
๐Ÿ’ฐ Only $19โ€“75 of Claude Code tokens per search
โšก Runs on stock vLLM, SGLang and llama.cpp, zero inference overhead

Blog, paper, code and checkpoints below ๐Ÿงต
View quoted post on X
1 replies 2 reposts 11 likes 811 views View on X
Yi Pan Verified account @conlesspan
Solid work from @KanZhu854772 on SSEngine ๐Ÿ”ฅ๐Ÿš€ Open-source prefill engine for GLM-5.3-Flash on four B200s, hitting 3.1โ€“3.2ร— higher request throughput than vLLM/SGLang on coding-agent traces ๐Ÿ“ˆ

Pipeline parallelism + tiered prefix cache + shortest-remaining-prefill-first scheduling really moved the needle here. Great example of how much specialized prefill stacks can outperform general serving frameworks.
Kan Zhu Verified account @KanZhu854772 3d
๐Ÿ”ฅ Serving coding agents well needs a high-performance prefill engine, but existing systems fall short.

๐Ÿš€ Introducing SSEngine-GLM5.3-Flash: an open-source prefill engine for GLM-5.3-Flash on four B200 GPUs. It combines pipeline parallelism, a tiered prefix cache across GPU memory, DRAM and SSD, and a shortest-remaining-prefill-first scheduler.

๐Ÿ“ˆ On real hardware, it sustains 3.1ร— (AgentX) and 3.2ร— (TraceLab) the request rate of any vLLM or SGLang setup we tested.

๐Ÿงต More in the thread below (1/8)
View quoted post on X
0 replies 1 reposts 13 likes 940 views View on X
Yi Pan Verified account @conlesspan
๐Ÿš€ Qwen, Kimi, GLM are all betting on linear attention. But it's still slow: the recurrent state is read & rewritten every token.

Meet LeapQuant: the first training-free method to make 8-bit or 4-bit recurrent states work.

๐Ÿ“ˆ Across Qwen, Kimi, and GLM models:
โ€ข Matches FP32 accuracy
โ€ข 4.25ร— faster linear attention kernels
โ€ข Up to 2ร— higher inference throughput
โ€ข Cuts memory by up to 56%
0:34
36 replies 127 reposts 871 likes 109K views View on X
Yi Pan Verified account @conlesspan
Really happy to have been part of this. Congrats to the whole team on FrontierCS 2! ๐Ÿš€๐ŸŽ‰
Qiuyang Mang Verified account @MangQiuyang Oct 22
FrontierCS 2 is our new benchmark for CS research & development.

Each task is based on a paper: we delete the contribution from the released code, and ask agents to outdo the original method, as judged by the authors' chosen metrics. Agents don't have access to the paper or web.
View quoted post on X
2 replies 0 reposts 11 likes 1.1K views View on X
Yi Pan Verified account @conlesspan
Excited to share our recent work on leveraging TorchDynamo in AI training and inference systems at @UWSyFi with @thepadawang at #PyTorchConf today! โšก๏ธ

We'll show how to use TorchDynamo beyond torch.compile, as a layer to accelerate, compare, and debug ML systems without rewriting the model.

๐Ÿ—“ 5:30โ€“5:55 PM PT
๐Ÿ› LL21A, San Jose Convention Center