@HLawrenceCS
Pinned post
We must choose both the tokens AND their order.
A joint distribution can be factored into a product of conditionals in any order, but this choice determines the available context at each step. Thus, some orders are easier to model in practice (arxiv.org/abs/2502.06768)
(2/7)View quoted post on X
New blogpost on tokenizing non-sequential data!
Language has sequential structure, which gave rise to the next-token prediction paradigm of LLMs. But we increasingly use LLMs for data without inherent order (e.g. images, molecules, sets). What does “next token” mean here?
(1/7)View quoted post on X
Excited to share that ASEN, our framework towards equivariant foundation models, is accepted to #ICLR2026!
📜paper: arxiv.org/abs/2603.19486
📍poster: Sat 4/25 PM, P3-#726
w/ Abhinav Goel, @dereklim_lzh, @HLawrenceCS, Stefanie JegelkaView quoted post on X
