Julius Berner

This account is verified. Learn more

@julberner

Research scientist @nvidia | postdoc @caltech | PhD @univienna | former research intern @MetaAI and @nvidia | views are my own

jberner.info Joined September 2018

393 Following 1.5K Followers 358 Posts

Latest posts 5

Pinned post

Julius Berner @julberner Oct 28

Tired of waiting for your diffusion models?

🚢 We just shipped a major FastGen update:

- new models: H3, LTX-2.5, LingBot-World, Cosmos 3, FLUX.2
- new methods: PDD, AYF, AnyFlow, Causal Forcing
- new features: multimodal distillation, image editing, camera conditioning, KV cache management, ComfyUI

🔗 Try it out here: github.com/NVlabs/fastgen

0:47

Julius Berner @julberner Nov 3

Great findings! FastGen (github.com/NVlabs/FastGen) has always initialized fake-scores from the teacher. One intuition: matched architectures may share approx. biases that cancel in the VSD difference.

While step-conditioned fake-scores/discriminators are more principled, we haven’t seen large gains in our earlier exp, so these results are encouraging! Alternatives include disjoint re-noising intervals or fake-score experts.

github.com GitHub - NVlabs/FastGen: NVIDIA FastGen: Fast Generation from Diffusion Models NVIDIA FastGen: Fast Generation from Diffusion Models - NVlabs/FastGen

Xianbang Wang Verified account @kevinxbwang2007 Nov 2

A bigger teacher should give you a better DMD student, right?
👉 NOT if your critic is small. The student gets worse, and can even collapse.

2 dead-simple critic fixes everyone overlooks:
1. Scale the critic with the teacher (joint scaling)
2. Tell the critic which generation step it's judging (STAC)

🌐 Project: peppaking8.github.io/critic-matters/
📄 Paper: arxiv.org/abs/2610.42135

View quoted post on X

Julius Berner @julberner Oct 21

Cool work and very close to our concurrent Parametric Trajectory Distillation (PTD): alan-lanfeng.github.io/PTD/

Some connections to previous works:
- Parallel Decoding Distillation (PDD): while PDD uses a picewise-linear parametrization of the student trajectory (fused during inference), SMD & PTD propose a basis expansion with learned coefficients (discarded at inference).
- Lagrangian Flow-Maps: PDD/PTD/SMD can be viewed as special cases, where the JVP can be computed in closed form, since the terminal time enters only through the student head.

Details on these connections in our papers:
- PTD: arxiv.org/abs/2610.11498
- PDD: arxiv.org/abs/2607.26004

alan-lanfeng.github.io Parametric Trajectory Distillation Few-step distillation for video and sound.

Ziyu Wan Verified account @raywzy1 Oct 18

Excited to introduce Simplified Matching Distillation (SMD) for few-step diffusion🧵

No auxiliary score model. No adversarial training. No generator modification.

You will get:
• Photorealistic appearance
• High-quality dynamics
• Memory-efficient training
• Mode covering with diverse outputs

SMD works well across images, video, joint audio–video, and 3D gen, validated across model scales up to 33B parameters.

Besides, SMD supports:
- Multi-teacher capability merging (like OPD in LLM)
- Cross-model capability transfer
- Bidirectional teacher to causal distillation

Full release. Paper, code, model and data.

Project Page: zjwfufu.github.io/SMD-Page/
Paper: arxiv.org/abs/2610.18472
Code: github.com/zjwfufu/SMD
Model Card & Data: huggingface.co/zjwfufu/SMD

Demo Video 👇

View quoted post on X

Julius Berner @julberner Oct 21

Parallel Decoding Distillation (PDD) is taking off 🛫

We’ve only released PDD end of July and it’s already adopted in:
- Alibaba’s MiniMax H3: huggingface.co/alibaba-pai/Mi…
- WorldPlay2’s causal distillation: worldplay2.github.io
- FastVideo’s FastH3 Omni: haoailab.com/blogs/fastvide…
- NVIDIA’s large language continuous diffusion model: arxiv.org/pdf/2610.02665
- Amazon’s generative forecasting model: arxiv.org/abs/2610.22358
- NVIDIA’s & MSR’s extensions to parametric student trajectories: alan-lanfeng.github.io/PTD & arxiv.org/pdf/2610.18472

We’ll be open-sourcing our implementation soon - stay tuned!

huggingface.co alibaba-pai/MiniMax-H3-Acc-LoRAs · Hugging Face Text-to-Video model. Download, explore files and run it on Hugging Face.

Julius Berner Verified account @julberner Jul 29

Having worked with many distillation methods for accelerating diffusion & flow models, I’m still blown away by the simplicity of Parallel Decoding Distillation (PDD).

1️⃣ State-of-the-art quality and diversity with variable NFE
2️⃣ First to scale trajectory-distillation to few-step video generation

PDD is about as cheap and simple as it gets:
🎯single, regression-based MSE loss using 1 student & 1 detached teacher forward
❌NO data required
❌NO adversarial loss or alternating training
❌NO JVP or finite-difference approx.
❌NO additional time-embedding or trainable models (e.g. fake score, discriminator)
❌NO stabilization tricks (e.g. normalization, clipping, warm-up)

View quoted post on X

Julius Berner @julberner Oct 20

The latest result of our amazing collaboration between FastVideo and FastGen: FastH3 Omni

- 12x faster reference-to-video (image, video, & voice references)
- distilled using our new combination of PDD & DMD

Blog: haoailab.com/blogs/fastvide…
PDD: x.com/julberner/stat…

haoailab.com FastVideo FastH3 Omni: Open Source Real-Time Minimax FastH3 Ref2VA on NVIDIA Blackwell, RTX Spark,... MiniMax H3 video and audio with up to approximately 12× lower end-to-end latency on eight GB200 GPUs, plus optimized NVFP4 QAD checkpoints for local AI.

Hao AI Lab Verified account @haoailab Oct 20

As promised, FastVideo open-sources @MiniMax_AI FastH3 Omni, training code included! 🚀

- Real-time gen @ 12x faster on NVIDIA Blackwell
- NVFP4/FP8 for Spark + RTX!
- ComfyUI day-0

Collab with @NVIDIAAI , Nuva Lab @nuvalab and Reactor @reactorworld

Open source must win! ♥️

View quoted post on X