A simple yet effective idea: enforcing *hypersphere (manifold) constraint* unlock the potential of continuous-token AR generative models. Our next-token, raster-order AR model hits FID 1.34 on ImageNet-256, setting a new SOTA for AR image generation. check huggingface.co/papers/2509.24…
Guolin Ke
@guolin_ke
Machine Learning & AI for Science. Created #LightGBM, #Graphormer, #UniFold, Uni-Mol, Uni-3DAR. Former Senior Researcher @MSFTResearch. Opinions are my own.
380 Following 1.2K Followers 113 Posts
Latest posts 5
🚀 Uni-3DAR’s crystal generation code and models are now publicly available.
✅ github.com/dptech-corp/Un…
Crystal generation has become a hot topic in AI4S, with most prior work leveraging diffusion models that typically employ fractional coordinates to scale atomic positions into the [0, 1] range.
In Uni-3DAR, however, we opted to directly use the atoms’ actual physical coordinates rather than fractional coordinates. Although this decision increases the training difficulty, it aligns the approach with other microscopic 3D tasks and better reflects the motivation of Uni-3DAR.
Based on this, we reformulate the crystal generation task as a two-frame 3D generation process: first, generating the lattice, and then generating the atoms within that lattice.
In addition to unconditional generation, we have also explored tasks such as crystal structure prediction (CSP) with specified element types and counts, as well as PXRD-guided crystal prediction. Traditional diffusion models usually require an additional encoder to process conditions and employ a CLIP-like mechanism for multimodal alignment. In contrast, Uni-3DAR adopts a unified autoregressive framework to seamlessly handle these conditions, thereby eliminating the need for an extra encoder.
Our method achieves impressive performance on crystal generation, particularly for conditional generation, significantly surpassing the previous state-of-the-art. An animation below better illustrates the crystal generation process. For more details, please refer to our paper: arxiv.org/pdf/2503.16278
✅ github.com/dptech-corp/Un…
Crystal generation has become a hot topic in AI4S, with most prior work leveraging diffusion models that typically employ fractional coordinates to scale atomic positions into the [0, 1] range.
In Uni-3DAR, however, we opted to directly use the atoms’ actual physical coordinates rather than fractional coordinates. Although this decision increases the training difficulty, it aligns the approach with other microscopic 3D tasks and better reflects the motivation of Uni-3DAR.
Based on this, we reformulate the crystal generation task as a two-frame 3D generation process: first, generating the lattice, and then generating the atoms within that lattice.
In addition to unconditional generation, we have also explored tasks such as crystal structure prediction (CSP) with specified element types and counts, as well as PXRD-guided crystal prediction. Traditional diffusion models usually require an additional encoder to process conditions and employ a CLIP-like mechanism for multimodal alignment. In contrast, Uni-3DAR adopts a unified autoregressive framework to seamlessly handle these conditions, thereby eliminating the need for an extra encoder.
Our method achieves impressive performance on crystal generation, particularly for conditional generation, significantly surpassing the previous state-of-the-art. An animation below better illustrates the crystal generation process. For more details, please refer to our paper: arxiv.org/pdf/2503.16278
Uni-3DAR generates 3D structures in a coarse-to-fine autoregressive manner. By dynamically expanding subcells with known positional priors, it captures 3D context better than traditional set-based AR models (e.g. a set of atoms) —boosting performance significantly.
Home Page: uni-3dar.github.io
Paper: arxiv.org/abs/2503.16278
Code: github.com/dptech-corp/Un…
To improve efficiency, previous methods typically adopt set-based approaches for 3D structures—for example, representing a molecular structure as a set of atoms. While such representations can work well for understanding tasks, they fall short in generation tasks. The key limitation is the lack of positional context: the model must predict the next atom’s 3D position from the entire 3D space, without any constraints. This contrasts sharply with autoregressive (AR) models in text or image domains, where the position of the next token is known during generation.
Our proposed Uni-3DAR addresses this challenge through a coarse-to-fine generation strategy. At each stage, the model determines which subcells will be expanded based on the current predictions. This allows us to know the positional information of each token ahead of time—mirroring the behavior of AR models in text and image generation. As a result, our method significantly outperforms previous set-based AR models.
As shown in Table 11 of our paper, removing these positional priors from the coarse-to-fine tokens leads to a significant performance drop. Even augmenting traditional AR models with token-level diffusion loss fails to recover the performance, underscoring the importance of positional guidance in 3D generation.
Home Page: uni-3dar.github.io
Paper: arxiv.org/abs/2503.16278
Code: github.com/dptech-corp/Un…
To improve efficiency, previous methods typically adopt set-based approaches for 3D structures—for example, representing a molecular structure as a set of atoms. While such representations can work well for understanding tasks, they fall short in generation tasks. The key limitation is the lack of positional context: the model must predict the next atom’s 3D position from the entire 3D space, without any constraints. This contrasts sharply with autoregressive (AR) models in text or image domains, where the position of the next token is known during generation.
Our proposed Uni-3DAR addresses this challenge through a coarse-to-fine generation strategy. At each stage, the model determines which subcells will be expanded based on the current predictions. This allows us to know the positional information of each token ahead of time—mirroring the behavior of AR models in text and image generation. As a result, our method significantly outperforms previous set-based AR models.
As shown in Table 11 of our paper, removing these positional priors from the coarse-to-fine tokens leads to a significant performance drop. Even augmenting traditional AR models with token-level diffusion loss fails to recover the performance, underscoring the importance of positional guidance in 3D generation.
The model, training scripts, and dataset for small molecule generation (QM9) with Uni-3DAR are now released!
Check it out on GitHub: github.com/dptech-corp/Un…
Check it out on GitHub: github.com/dptech-corp/Un…
The world is 3D—and many scientific laws are rooted in 3D space.View quoted post on X
If we want a world model, 3D structure modeling is essential.
Yet today, there’s no universal 3D representation—not across scales, not even within microscopic systems.
We present Uni-3DAR, a unified 3D framework with:
🔹 Handles 3D structures from micro to macro scales
🔹 Coarse-to-fine octree + 2-level compression
🔹 Fine-grained 3D patch tokenization
🔹 Masked next-token prediction for autoregressive modeling
🔹 Better and faster than diffusion
home page: uni-3dar.github.io
paper: arxiv.org/pdf/2503.16278
The world is 3D—and many scientific laws are rooted in 3D space.
If we want a world model, 3D structure modeling is essential.
Yet today, there’s no universal 3D representation—not across scales, not even within microscopic systems.
We present Uni-3DAR, a unified 3D framework with:
🔹 Handles 3D structures from micro to macro scales
🔹 Coarse-to-fine octree + 2-level compression
🔹 Fine-grained 3D patch tokenization
🔹 Masked next-token prediction for autoregressive modeling
🔹 Better and faster than diffusion
home page: uni-3dar.github.io
paper: arxiv.org/pdf/2503.16278
If we want a world model, 3D structure modeling is essential.
Yet today, there’s no universal 3D representation—not across scales, not even within microscopic systems.
We present Uni-3DAR, a unified 3D framework with:
🔹 Handles 3D structures from micro to macro scales
🔹 Coarse-to-fine octree + 2-level compression
🔹 Fine-grained 3D patch tokenization
🔹 Masked next-token prediction for autoregressive modeling
🔹 Better and faster than diffusion
home page: uni-3dar.github.io
paper: arxiv.org/pdf/2503.16278