Twitxy Saved

Yonggan Fu

Verified account This account is verified. Learn more

@YongganFu

Senior Research Scientist @ NVIDIA Research; PhD @ Georgia Institute of Technology
Santa Clara, California yongganfu.com Joined October 2024
95 Following 187 Followers 19 Posts

Latest posts 5

Pinned post

Yonggan Fu Verified account @YongganFu
🚀 Check out our Nemotron-Labs-Diffusion model!

We have been wondering about the true promise of diffusion LMs, especially when competing with strong AR models in terms of accuracy and with MTP methods in terms of efficiency.

💡 Nemotron-Labs-Diffusion is an important step toward answering this question, delivering a tri-mode LM that unifies AR, diffusion, and self-speculation (diffusion drafts, AR verifies) decoding within a single model.

🌟 Core Insights
🔸 AR and diffusion objectives can be mutually beneficial and harmonized within a single model.
🔸 Self-speculation, enabled by joint AR/diffusion training, can outperform Eagle3 in throughput.
🔸 Diffusion shows strong long-term potential for parallel decoding under an optimal sampler.

📊 Model performance: Our 8B/14B models match or exceed Qwen3-8B/14B accuracy while generating 6× more tokens per forward pass, leading to a 4× speedup in SGLang on GB200.

🤗 𝗛𝗙 𝗖𝗼𝗹𝗹𝗲𝗰𝘁𝗶𝗼𝗻 (3B/8B/14B base + instruct and 8B VLM):
lnkd.in/gej2VfN3

📰 𝗧𝗲𝗰𝗵 𝗥𝗲𝗽𝗼𝗿𝘁:
lnkd.in/gvS2Pwe8
0:38
3 replies 20 reposts 120 likes 185K views View on X
Yonggan Fu Verified account @YongganFu
See you next Mon!
Discrete Diffusion Reading Group Verified account @diffusion_llms Jun 25
📢June 29 (Mon): Nemotron-Labs-Diffusion: Unifying AR, Diffusion, and Self-Speculation

🌟Nemotron-Labs-Diffusion is a tri-mode language model (LM) that unifies AR, diffusion, and self-speculation decoding within a single architecture.

💪Trained with a joint AR-diffusion objective, Nemotron-Labs-Diffusion can switch modes to sustain high throughput across deployment settings and concurrency levels. Their study shows that

1️⃣AR and diffusion objectives are complementary: diffusion improves lookahead planning, while AR provides left-to-right linguistic priors.

2️⃣In self-speculation mode, diffusion drafts while AR verifies, outperforming multi-token prediction (MTP) methods in both acceptance rate and real-device efficiency.

3️⃣A speed-of-light analysis further demonstrates diffusion’s long-term potential, with up to 76.5% more tokens per forward pass than self-speculation under an optimal sampler.

📈Scaling to 3B, 8B, and 14B parameters, the Nemotron-Labs-Diffusion family, including base, instruct, and vision-language models, consistently outperforms state-of-the-art open-source AR and diffusion LMs in both accuracy and speed.

⚡For example, Nemotron-Labs-Diffusion-8B decodes 6× more tokens per forward than Qwen3-8B with comparable accuracy, translating to 4× higher throughput on SPEED-Bench with SGLang on a GB200 GPU.

This Monday, Yonggan Fu (@YongganFu) from NVIDIA Research will present Nemotron-Labs-Diffusion: Unifying AR, Diffusion, and Self-Speculation.
View quoted post on X
0 replies 2 reposts 14 likes 1.6K views View on X
Yonggan Fu Verified account @YongganFu
Masked diffusion naturally suffers from a left-to-right tendency and is typically restricted to small block sizes due to the information void across denoising steps. Soft mixing of candidate embeddings could be a promising mechanism to pass information across steps, enabling larger block sizes and greater parallelism.
Google Gemma Verified account @googlegemma Jun 10
Meet DiffusionGemma!

An experimental open model that explores a fast approach to text generation, released under an Apache 2.0 license.

Moving beyond sequential, token-by-token processes to generate entire blocks of text simultaneously. Here’s what’s new with DiffusionGemma: 👇
View quoted post on X
Yonggan Fu Verified account @YongganFu
The Nemotron-Labs-Diffusion family has reached 503K downloads on Hugging Face (huggingface.co/collections/nv…) in the past month!

We provide a deployment guide for running Nemotron-Labs-Diffusion with SGLang in our GitHub repository:
github.com/NVlabs/Nemotro… (welcome to star ⭐)

The attached demo was on an NVIDIA DGX Spark (your personal AI computer). You can easily reproduce it with a nice chat interface by following our GitHub guide!

Tech report: d1qx31qr3h6wln.cloudfront.net/publications/N…
0:24