Twitxy Saved

Dmitry Krotov

Verified account This account is verified. Learn more

@DimaKrotov

Co-Founder and CEO at Dynamical Mind. Physics, Associative Memory, Generative AI. Formerly: @MITIBMLab, @IBMResearch, @the_IAS, @Princeton.
Cambridge, MA dmitrykrotov.com Joined December 2011
995 Following 5K Followers 422 Posts

Latest posts 5

Dmitry Krotov Verified account @DimaKrotov
It’s astonishing how we manage to make running some of America’s most iconic companies so unnecessarily complicated.

Of course, being romantically involved with someone suspected of working for a foreign intelligence raises serious national security concerns.

But there’s a profound irony here: the decision to push out an extraordinary CEO like Jensen Huang may ultimately do more damage to America’s technological leadership than the alleged espionage ever did.
2 replies 4 reposts 52 likes 3.1K views View on X
Dmitry Krotov Verified account @DimaKrotov
Our new paper in Nature Communications reveals an interesting connection between generative diffusion models and Associative Memory.

It identifies 3 phases of generated samples:
👉 Memorization: reproducing training examples verbatim.
👉 Spurious states: discovering new patterns between stored memories, while maintaining a well-defined basin of attraction around those samples.
👉 Generalization: generating genuinely novel samples with flat energy landscape around them.

The surprising twist? What was traditionally considered a flaw in Associative Memory — spurious states — may actually be the first sign of generative capabilities.

It was fun to work on this project with @baophamhq, @gaboraya, Matteo Negri, @mj_zaki, and @LucaAmb, and I am happy to see it published!

Paper: nature.com/articles/s4146…
5 replies 46 reposts 256 likes 16K views View on X
Dmitry Krotov Verified account @DimaKrotov
Can a neural network learn ~2^100 LDPC (Low-Density Parity-Check) codewords from just 10-20 examples?

👉 Grokking in LLMs and memorization-to-generalization transition in diffusion models are related phenomena — the model learns true process of data generation from a small number of training samples.

👉 It is hard to study this phenomenon analytically on natural datasets.

👉 LDPC codeword generation is a precise mathematical process in which the codewords are constructed as solutions to parity check equations.

👉 We train a Dense Associative Memory with a tiny fraction of codewords from the full vocabulary, without giving the network the generation algorithm itself. The model manages to learn the entire vocabulary on its own. It turns out, you only need log(n) samples to learn the entire space of 2^n codewords.

👉 The network only needs approximately P=10-20 training examples to correctly learn the full vocabulary containing ~2^100 codewords.

Our new paper with Andy Lucas: arxiv.org/abs/2610.27384.

Let us know what you think!
6 replies 16 reposts 119 likes 5.2K views View on X
Dmitry Krotov Verified account @DimaKrotov
In 2016: “AI will replace radiologists within 5 years”
2026: Demand for radiologist jobs is high. We still need them!
Now: “AI has a 25% chance of causing human extinction”

From replacing jobs to replacing humanity — the predictions keep getting bigger. The evidence? Not so much.

Why are we doing this to ourselves?