Super fun project that shows that even (masked) discrete dLLMs can generate multiple tokens in-parallel just fine!
Diffusion LLMs are meant to be fast ⚡️: they can decode many tokens in parallel in a single step.View quoted post on X
The catch: push the parallelism too far and generation quality drops sharply.
We introduce a simple fix: Temporal Self-Distillation (TSD): the model learns from its own later, more-confident predictions, using only the rollout a verifier marks correct 👇🧵