Twitxy Saved

G Bruno De Luca

@gbruno_dl

MIAI Research Chair @ LAPTh, Annecy.
Previously postdoc @ IFT Madrid & Stanford Institute for Theoretical Physics. Phd @ Milano-Bicocca
gbdl.github.io
Joined September 2026
0 Following 95 Followers 3 Posts

Latest posts 1

G Bruno De Luca @gbruno_dl
Can we bootstrap intelligence? Probably! arxiv.org/abs/2609.30063

Thanks @AdityaCowsik @KfirDolev @michaelyli_ @ANourya @noahdgoodman @YoavLevine for the great work!
arxiv.org Self-Play Pretraining with Zero Data Advances in language modeling have been driven by scaling pretraining on ever more data. Yet, the training data is still largely curated on the model's behalf. A more general approach to...
Michael Y. Li Verified account @michaelyli_ Sep 25
Can an LM, starting from random init (!!), learn to generate all of its pretraining data?

Introducing Self-Play Pretraining with Zero Data. Two models start from random initialization: a generator proposes programs for a universal Turing machine and a learner trains on their outputs. We never train on any real data, but see predictable scaling on natural datasets: zero-shot val loss on images, text, audio, and melodies decreases predictably with self-play compute. And the learner develops in-context learning capabilities.

A fun proof-of-concept, co-led with @AdityaCowsik and @KfirDolev and co-authors @gbruno_dl, @ANourya @noahdgoodman, and @YoavLevine.
View quoted post on X