Pinned post
The belief that language modeling in continuous space is hard and unrewarding is fading, as continuous diffusion and flow LMs become competitive while offering easier control and faster generation. Our survey comprehensively covers the field's history and accumulated knowledge!
New survey on Continuous Diffusion and Flow Language Models is out! ๐๐ฌ
One of my first papers, back in early 2025, was on continuous simplex flows. After working on discrete diffusion, I wanted to try controllable sequence generation with flow matching, which naturally operates in continuous spaces. But at the time, I found them much harder to train well than discrete diffusion. ๐ฅฒ
2026 has changed this picture completely! ๐ Itโs been super exciting to watch continuous LMs take off, but with the rapid advancements, itโs also becoming harder to keep up with all the new work across pre-training, few-step generation, guidance, and scaling ๐ฅน
So together with @chandavidlee, @FEijkelboom, and @jw9730, we put together a survey that consolidates the continuous LM literature and its key insights into 37 pages. It covers everything from the early work before the so-called โwinterโ of continuous LMs to the wave of new papers in 2026 ๐
More in thread! ๐งตView quoted post on X
