Pinned post
Stefano Martiniani
Verified account This account is verified. Learn more@SteMartiniani
Co-Founder of Matterstack | Professor at New York University
1.6K Following 1.5K Followers 1.5K Posts
Latest posts 5
Look what I found walking by Washington Square Park this morning, a highlight of “the discovery of Gyromorphs, a material that combines the seemingly incompatible properties of liquids and crystals, and that performs better than any other known structure in blocking light from all incoming angles”. Thanks NYU @ArtsandScience for the nice surprise!
🚨 We introduce a new attack capable of efficiently extracting training data from diffusion models. 🚨
The attack is completely unsupervised. It does not rely on the text paired with the training images, prompt engineering, training-set lookup, gradients, or access to the model weights. If you have access to the model input you can audit it. All on a single GPU.
🔁 All you need is the ability to run the diffusion process in cycles. Take an image. Add noise partway through the diffusion process. Denoise it. Feed the result back in. Repeat.
At first, the trajectory wanders through generic samples. The images change from cycle to cycle, and they look like ordinary outputs from the model rather than memorized training examples.
Then something striking happens: the trajectory gets trapped. 🪤
It enters a memorized state and stays there for hundreds of cycles. After each noise–denoise cycle, the image comes back essentially unchanged. Those long plateaus in similarity are the signal. 💡
We call these *ultrastable memories.* Many correspond to memorized training content: stock-photo templates, product images, logos, watermarks, web-crawl artifacts, and repeated e-commerce scenes.
Cyclic denoising gives us a new way to audit diffusion models for privacy, copyright, and trademark leakage. If a model can be driven into protected images, brand marks, watermarked assets, or commercial templates without the training caption, the usual story about “just learning the distribution” becomes harder to maintain.
The punchline: memorization in diffusion models is not confined to a few lucky prompts or isolated examples. It is extensive. And with the right cyclic drive, it becomes directly observable. 🔁 👀
👨🔬 Work led by Rishab Sharma @ssh_rishabh at NYU
📖 Paper: R. Sharma, S. Martiniani, "Cyclic Denoising Reveals Ultrastable Memories in Diffusion Models", lnkd.in/gfEYezGu
The attack is completely unsupervised. It does not rely on the text paired with the training images, prompt engineering, training-set lookup, gradients, or access to the model weights. If you have access to the model input you can audit it. All on a single GPU.
🔁 All you need is the ability to run the diffusion process in cycles. Take an image. Add noise partway through the diffusion process. Denoise it. Feed the result back in. Repeat.
At first, the trajectory wanders through generic samples. The images change from cycle to cycle, and they look like ordinary outputs from the model rather than memorized training examples.
Then something striking happens: the trajectory gets trapped. 🪤
It enters a memorized state and stays there for hundreds of cycles. After each noise–denoise cycle, the image comes back essentially unchanged. Those long plateaus in similarity are the signal. 💡
We call these *ultrastable memories.* Many correspond to memorized training content: stock-photo templates, product images, logos, watermarks, web-crawl artifacts, and repeated e-commerce scenes.
Cyclic denoising gives us a new way to audit diffusion models for privacy, copyright, and trademark leakage. If a model can be driven into protected images, brand marks, watermarked assets, or commercial templates without the training caption, the usual story about “just learning the distribution” becomes harder to maintain.
The punchline: memorization in diffusion models is not confined to a few lucky prompts or isolated examples. It is extensive. And with the right cyclic drive, it becomes directly observable. 🔁 👀
👨🔬 Work led by Rishab Sharma @ssh_rishabh at NYU
📖 Paper: R. Sharma, S. Martiniani, "Cyclic Denoising Reveals Ultrastable Memories in Diffusion Models", lnkd.in/gfEYezGu
0:50
💎 OMatG just became the #1 generative model for materials discovery.
Independent results on LeMat-GenBench ⚛️🤗, the most comprehensive benchmark for crystal generation, just dropped. A huge thanks to the incredible team that put this leader board together.
Here is how we compare to Microsoft's MatterGen:
SUN (stable + unique + novel): 1% vs. 0.2%, a 5x improvement
MSUN (metastable): 18% vs. 15.0%
RMSD (structural precision): 0.076 vs. 0.39 Å, a 5x improvement
Effort led by @PhilippHoellmer, @MMartirossyan, Thomas Egg at @nyuniversity. Model trained on Empire AI. OMatG is open source and open weights.
If you want to learn more about OMatG and the broader OMatG family of models, reach out.
Independent results on LeMat-GenBench ⚛️🤗, the most comprehensive benchmark for crystal generation, just dropped. A huge thanks to the incredible team that put this leader board together.
Here is how we compare to Microsoft's MatterGen:
SUN (stable + unique + novel): 1% vs. 0.2%, a 5x improvement
MSUN (metastable): 18% vs. 15.0%
RMSD (structural precision): 0.076 vs. 0.39 Å, a 5x improvement
Effort led by @PhilippHoellmer, @MMartirossyan, Thomas Egg at @nyuniversity. Model trained on Empire AI. OMatG is open source and open weights.
If you want to learn more about OMatG and the broader OMatG family of models, reach out.
