Twitxy Saved

Mo Piening

@moritz_piening

Researcher | Tweeting Novice
U Göttingen / TU Berlin
Interested in the geometry of generative models

moepien.github.io
Joined November 2022
72 Following 39 Followers 16 Posts

Latest posts 5

Mo Piening @moritz_piening
Great follow-up to great work :)

Super interested in a deeper understanding of optimal stage-1 pretraining across datasets, beyond the empirical findings!
Jaeyeon (Jay) Kim Verified account @Jaeyeon_Kim_0 Nov 4
Since releasing Priming Models, we’ve received great questions. Here’s a short follow-up with new experiments and release updates!

TL;DR: Diffusion policy setup / (# real samples) vs. Final FID / T2i checkpoints released / new blog post from @moritz_piening!
View quoted post on X
Mo Piening @moritz_piening
Idea by @Jaeyeon_Kim_0 and @parkwbin03 was so intruiging that I had to play around a bit.

Check out experiments with Priming Models on CIFAR-10, MNIST, and Fashion-MNIST:
moepien.github.io/blog/better-pr…

TL;DR: Stage-1-Init is very sensitive, but optimal transport can help a bit! :)
Jaeyeon (Jay) Kim Verified account @Jaeyeon_Kim_0 Oct 15
Today, we introduce Priming Models, a new paradigm for training one-step generative models from scratch.

No diffusion teacher. No flow map. No CFG. No JVP.

We believe this work will challenge how we think about building one-step models.

Joint work with @sitanch, @du_yilun, @parkwbin03, Bohyung Han.

🔥 1.19 FID one-step on ImageNet 256×256 while using substantially less compute, e.g., 12X less than Drifting-B, 6X less than Drifting-L
🎨 Scales to text-to-image generation (GenEval 0.82)
🤖 World modeling on RT-1 / Bridge-V2

Priming Model is built on one core message: [Reconstruction pretraining --> Distributional post-training].
View quoted post on X
0 replies 7 reposts 50 likes 3.7K views View on X