Twitxy Saved

Gabriel Raya

@gaboraya

PhD Candidate Machine Learning @TilburgU @TUeindhoven @SonyAI_global MSc Data Science @radboud_uni @radboudumc - Deep Generative Models
The Netherlands gabrielraya.com Joined May 2014
304 Following 167 Followers 168 Posts

Latest posts 5

Gabriel Raya @gaboraya
It was a real pleasure working with such brilliant collaborators!

We explore how generative behavior in diffusion models emerges from associative memory. The energy landscape first favors memorized attractors; as data grows, spurious states form near the onset of generalization
Dmitry Krotov Verified account @DimaKrotov Nov 2
Our new paper in Nature Communications reveals an interesting connection between generative diffusion models and Associative Memory.

It identifies 3 phases of generated samples:
👉 Memorization: reproducing training examples verbatim.
👉 Spurious states: discovering new patterns between stored memories, while maintaining a well-defined basin of attraction around those samples.
👉 Generalization: generating genuinely novel samples with flat energy landscape around them.

The surprising twist? What was traditionally considered a flaw in Associative Memory — spurious states — may actually be the first sign of generative capabilities.

It was fun to work on this project with @baophamhq, @gaboraya, Matteo Negri, @mj_zaki, and @LucaAmb, and I am happy to see it published!

Paper: nature.com/articles/s4146…
View quoted post on X
0 replies 1 reposts 15 likes 796 views View on X
Gabriel Raya @gaboraya
Another one on the continuous diffusion language saga
Zhihan Yang Verified account @zhihanyang_ May 19
📢Excited to share our new paper:

Continuous Diffusion Scales Competitively with Discrete Diffusion for Language

We introduce RePlaid 🌊, a continuous diffusion language model (DLM) with
🏅Discrete likelihood bound
🏅Scaling laws competitive with SOTA discrete DLMs

How? Dive in👇[🧵1/12]

Paper: arxiv.org/abs/2605.18530

Work done with my amazing collaborators: @WeiGuo01 @ShuibaiZ69721 @ssahoo_ @YongxinChen1 @ArashVahdat @MardaniMorteza @jwthickstun
View quoted post on X
0 replies 3 reposts 10 likes 1.4K views View on X
Gabriel Raya @gaboraya
At the core of efficient diffusion is a simple question: where is information actually resolved?

The entropy profile answers this, guiding training effort toward the regions where structure is formed. Great to see this perspective used for continuous bitstream language diffusion
Luca Ambrogioni Verified account @LucaAmb May 11
1/?) As promised to Sander Dieleman (@sedielem), we’re finally excited to share:

Towards Closing the Autoregressive Gap in Language Modeling via Entropy-Gated Continuous Bitstream Diffusion

We show that continuous diffusion can achieve very strong language modeling performance when operating directly on bitstreams, outperforming masked and uniform diffusion baselines, and essentially matching autoregressive models under our evaluation settings.
View quoted post on X