Twitxy Saved

Mayank Mishra

Verified account This account is verified. Learn more

@MayankMish98

Doing philosophy with @tri_dao, @profjoeyg, @istoica05
Berkeley mayank31398.github.io Joined June 2017
787 Following 2.2K Followers 358 Posts

Latest posts 5

Pinned post

Mayank Mishra Verified account @MayankMish98
We pretrained a 2.3B MoE (360M active) Hybrid Mamba-2 that lands within a few points of Llama-3.2-3B using <1% of its pretraining FLOPs.

No dedicated cluster. The run hopped between H100s, A100s, V100s (yes, V100s) and TPU v5p/v6e on a single codebase.

Meet Rigel 🧵
23 replies 79 reposts 635 likes 87K views View on X
Mayank Mishra Verified account @MayankMish98
Checkout LeapQuant from @conlesspan and @HaochengXiUCB!
Training free quantization for recurrent states of linear attention 🚀🚀🚀
Yi Pan Verified account @conlesspan Nov 11
🚀 Qwen, Kimi, GLM are all betting on linear attention. But it's still slow: the recurrent state is read & rewritten every token.

Meet LeapQuant: the first training-free method to make 8-bit or 4-bit recurrent states work.

📈 Across Qwen, Kimi, and GLM models:
• Matches FP32 accuracy
• 4.25× faster linear attention kernels
• Up to 2× higher inference throughput
• Cuts memory by up to 56%
View quoted post on X
0 replies 2 reposts 11 likes 996 views View on X
Mayank Mishra Verified account @MayankMish98
Check out FrontierCS 2 from @MangQiuyang 🚀
A new benchmark for CS research!!
Qiuyang Mang Verified account @MangQiuyang Oct 22
FrontierCS 2 is our new benchmark for CS research & development.

Each task is based on a paper: we delete the contribution from the released code, and ask agents to outdo the original method, as judged by the authors' chosen metrics. Agents don't have access to the paper or web.
View quoted post on X