@ArnavChavan6
Pinned post
Can flow language models scale to billions of parameters and handle real generative tasks?
Introducing 🌊 Lafa (Large flow language model), a flow LM scaled up to 8B that matches AR quality, with 🚀 2.5× AR throughput at 7B.
Co-led with @ArnavChavan6 @UdbhavBamba.
Joint work with @wognsfjq96 @hongseu33 @AdtRaghunathan @jw9730View quoted post on X
Model competition is heating up, and distillation is 🔥🔥View quoted post on X
Distillation detection is hard, so watermarking can help make it detectable.
But are existing watermarks robust?
We find two ways to break them: distill a model, then distill that model again; or distill only selected tokens that matter.
Then what do we do? We introduce SubWM, which outperforms prior methods and is more robust to laundering.
🏆 “Large Flow Language Models” 🏆 is now an officially certified arXiv Banger.View quoted post on X
Congrats @ArnavChavan6, @chandavidlee, @UdbhavBamba, @wognsfjq96 and @hongseu33, et al.
Excited to present our new work on distillation detection: Subliminal Watermarks (SubWM). SubWM is much more robust compared to existing distillation detectors, is able to distinguish different watermarking keys, and also works reliably for out-of-domain prompts!
Blog: subliminal-watermarks.github.io
🧵View quoted post on X
