Benjamin Minixhofer (@bminixhofer)

Pinned post

Benjamin Minixhofer @bminixhofer Dec 15, 2025

We are releasing Bolmo today!

Bolmo is the best byte-level model so far. It comes close to and sometimes surpasses Olmo 3.

Bolmo also performs competitively in terms of speed & is fully open.

I was skeptical of byte-level models for a long time but I finally switched camps🧵

Ai2 Verified account @allen_ai Dec 15, 2025

Introducing Bolmo, a new family of byte-level language models built by "byteifying" our open Olmo 3—and to our knowledge, the first fully open byte-level LM to match or surpass SOTA subword models across a wide range of tasks. 🧵

View quoted post on X

Benjamin Minixhofer @bminixhofer Oct 28

Very exciting new work on H-Net! Some notes:

1. The scaling analyses look super good. Definitely makes me more optimistic about H-Net style models taking over in the medium-term.

2. 'In-context tokenization' also appeared in BLT so this does not seem like a very surprising new finding to me (e.g. Figure 9 in the BLT paper).

3. At this stage of the research on hierarchical byte-level models I am a bit disappointed by work being kept private ('We plan to release some of these architectures to the community over time' in the blog post), although of course it is understandable that there are financial incentives at play. It's very early and would be so cool to get more people involved! There are so many things to figure out. Hopefully @cartesia can release more details soon.

Albert Gu Verified account @_albertgu Oct 28

I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.

Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?

Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.

View quoted post on X

Benjamin Minixhofer @bminixhofer Oct 23

Come see our work @emnlpmeeting !

CambridgeLTL @CambridgeLTL Oct 23

The @Cambridge_Uni Language Technology Lab is heading to Budapest for #EMNLP2026 next week! 🇭🇺

Our very own @annalkorhonen will deliver a keynote, "AI for the Many: Starting from the World, Not the Model" 🗓️ October 27, 14:00–15:00

Full sched below, come say hi! 👋

View quoted post on X

Benjamin Minixhofer @bminixhofer Oct 20

It's great to see more competitive linear RNN variants come out of @jkulinz 👇

Congrats @anamariarp17 @zlymi_levi and team!

Anamaria-Roberta Hartl Verified account @anamariarp17 Oct 20

New paper 🧵 KronLSTM: a new member of the xLSTM family, built on the Kronecker Delta Rule.
✏️ less interference than delta-rule updates
📏 provably bounded memory, no normalizer state
📈 competitive with, often ahead of, (Gated) DeltaNet, especially at long context

View quoted post on X

Benjamin Minixhofer @bminixhofer Oct 13

I somehow missed this when it came out.

How to increase the malleability of LLMs is super interesting.

Also a good example of the (lack of) correlation between base model perf and perf after post-training.

Benjamin Minixhofer

Latest posts 5