Dan Fu

Together AI

@realDanFu

Founding SVP of Research @togethercompute
Assistant Professor @ucsd_cse

Looking for talented kernel engineers and performance engineers!

danfu.org Joined September 2019

246 Following 8.6K Followers 967 Posts

Latest posts 5

Pinned post

Dan Fu @realDanFu Aug 19, 2024

Excited to share that I will be joining UCSD CSE as an assistant professor in January 2026!

I'll be recruiting PhD students from the 2024 application pool - if you're interested in anything ML Sys/efficiency/etc please reach out & put my name on your application!

Until then I'll be finishing up some requirements at Stanford (long story...) and hanging out at @togethercompute. Stay tuned for more!

Dan Fu @realDanFu 3d

Super excited to (officially) share Together Compiler with the world! This is our opinionated take at @togethercompute about how to scale inference in an increasingly polymorphic world (many models, archs, traffic, hardware).

Deployed across our inference fleet today 🚀 1/

Simran Arora Verified account @simran_s_arora 3d

Introducing the Together Compiler: continual compilation for AI inference! I’m super proud to finally share this work from our research team! ✨

Today’s inference is “Polymorphic” or shape-shifting, it doesn’t sit still. On OpenRouter, 80% of traffic goes to models released in the last three months and the top ten most popular open models each use a different architecture. Our customers constantly change their usage behaviors. Together Compiler leverages agents to continually optimize our deployments immediately as these changes occur.

Together Compiler research highlights include:
1. 252x better throughput: TKCC lets agents try 252x more optimization experiments per hour compared to prior agent inference optimization systems.
2. State-of-the-art TPS across InferenceX workloads we tried: TKCC outperforms the best InferenceX engine across all tested settings in just 24 hours & 2 nodes per compile. Including 1111 TPS on MiniMax M3.1 B200 and 1205 TPS on DeepSeek v4.1 Flash GB300 at batch size 1. NVIDIA, AMD, Trainium2.
3. Continuous recompilation: TKCC currently serves many trillions of production tokens per day and adapts to traffic shifts within just a few hours of recompilation.
4. Bitter lessons: TKCC’s design builds on 2+ years of building frameworks to accelerate AI Inference, starting from ThunderKittens and Megakernels. Compilers have always used abstractions and IRs, but we show that agentic compilers now reach higher performance ceilings without abstractions.

From an incredible team @reyna_abhyankar @dylan__lim @realDanFu @togethercompute!!

View quoted post on X

Dan Fu @realDanFu 4d

PKB accepted to #neurips2026, go chat with @opengroundsFX and @simonguozirui in Sydney if you’re interested!

Fun fact - arxiv actually rejected this paper, so my accept rate on arxiv this cycle is worse than NeurIPS :)

Grateful for @askalphaxiv for an alternative!

Willy Chan @opengroundsFX 4d

Proud to announce that ParallelKernelBench (alphaxiv.org/abs/2606.paral…) has been accepted to NeurIPS 2026! 🎉

If you are interested in the future of multi-GPU kernels, come say hi!

Huge thanks to my amazing collaborators @asplencmnt @simonguozirui @simran_s_arora @realDanFu
View quoted post on X

Dan Fu @realDanFu Oct 29

Introducing NyoomFloat12, a hardware-friendly lossless 12-bit quant from BF16 (accepted to #neurips2026)!

In BF16 LLMs four of the bits that are the same for 99.7% of the weights. So you can compress BF16 -> 12-bit with minimal overhead.

Check out @__sylvie_l__'s thread! 1/

Sylvie L @__sylvie_l__ Oct 29

Lossless weight compression has saved LLMs GPU memory, but not time. Decoding costs more than it saves.
NyoomFloat12 (NF12) is a lossless 12-bit BF16 format with decode cheap enough to speed up inference. Qwen3-32B: 1.23× faster token generation, up to 2.54× rollout throughput🧵

View quoted post on X

Dan Fu @realDanFu Oct 28

I had a great time at this launch event, thanks for having me @StepFun_ai!

StepFun @StepFun_ai Oct 28

Rain couldn't stop the builders.

Great conversations at our Step 5 Preview Launch Night with @togethercompute!

From long-horizon agents and tool-calling reliability to inference costs and what AI breakthroughs mean for the next wave of startups.

Huge thanks to our panelist Dan Fu (@realDanFu · @togethercompute), Mar Hershenson (@MarHershenson · @pearvc), Saoud Rizwan (@sdrzn · @cline), and Joey Wang (@harvey), and everyone who joined us!

More to come.

View quoted post on X