Tomasz Limisiewicz

@TomLimi

Postdoctoral researcher at @meta Fair and @uwnlp , Interested in going into the inner workings of neural networks, multilingualism, and fairer NLP (he/him)

Seattle Joined September 2021

536 Following 887 Followers 367 Posts

Latest posts 5

Pinned post

Tomasz Limisiewicz @TomLimi May 4

We present Compute Optimal Tokenization! 🔡
Common in LLM scaling works stick to one tokenizer, sweeping data/model size.
But what happens when we control the tokenizer’s compression rate (bytes/token)?
Here we sweep tokenizers, params, and data across compute budgets: [1/N]

0:14

Tomasz Limisiewicz @TomLimi 5d

Amazing results! A great case for byte-based and latent-tokenized models like H-Net!
Still, most LLM research and all known frontier LLMs use subwords.
Here is my list of crucial research directions pushing us toward non-subword frontier models:
1/5

Albert Gu Verified account @_albertgu Oct 28

I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.

Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?

Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.

View quoted post on X

Tomasz Limisiewicz @TomLimi Oct 27

Szia 👋🇭🇺
Happy to be in Budapest for EMNLP.

I'll be sharing few lessons learned on byte language models, latent tokenization at tomorrow’s Tokenization Tutorial (in P2 from 9 -12:30).

tokenization-tutorial.github.io/tutorial/

Tomasz Limisiewicz @TomLimi Oct 10

There are bigger workshops out there, but @tokshopMeeting is the friendliest one!

Tomasz Limisiewicz @TomLimi Oct 10

@clara__meister dropping hot takes on tokenization in first sentence already 🔥

Tokenization Workshop (TokShop) @COLM2026 @tokshopMeeting Oct 10

Finally, we have an amazing lineup in the panel moderated by Yuval!

View quoted post on X