Thomas Fel

This account is verified. Learn more

@thomas_fel_

Interpretability, Visual Intelligence @GoodfireAI.
Prev: @Harvard, @Google, @BrownUniversity (@tserre lab). Crêpe lover.

San Francisco, CA thomasfel.me Joined February 2017

1K Following 3.8K Followers 1.8K Posts

Latest posts 5

Pinned post

Thomas Fel @thomas_fel_ Jul 7

Our work on Block-Sparse Featurizer is out 🧊 :)

We revive an old idea from the structured sparsity literature and use it to carve activation space into meaningful regions.

It's a first concrete answer to the question our concept manifolds work left open ! :)

Goodfire @GoodfireAI Jul 7

If models think in shapes, our tools should too.

Our latest research: Block-Sparse Featurizers (BSFs), a new way to find concepts in model activations - using multidimensional “blocks” instead of single directions. (1/9)

View quoted post on X

Thomas Fel @thomas_fel_ 4d

It is a very important paper imo

Our theory of neural representation (overcomplete, distributed) makes few testable predictions, and interference is one of them.

To know if it’s true we have to check, and this paper does it so well, it is a beautiful piece of science

Naomi Saphra @nsaphra 4d

@thebasepoint This one but we're about to drop a new version with some stuff added for ICLR, like more models and random seed experiments that control for non-geometric explanations. (And we're doing followup to expand outside of these three settings.) arxiv.org/abs/2606.13934
View quoted post on X

Thomas Fel @thomas_fel_ 6d

A science of post-training can be built on interpretability.

Models know when they reward hack, and we see it, yet training ignores it. We propose Mechanistic Shaping: using the model’s own concepts to write training rules.

In vision, DPO & RL, and its effects on monitoring 🧵

0:14

Thomas Fel @thomas_fel_ Oct 15

When Nick shared his work months ago, I was skeptical.

I dug in with @CurtTigges and @nickcammarata to prove them wrong, building toy models with known DGPs, then bigger models, and ended up really convinced

Beautiful work that changed how I think about interp 🥹

👑

Nick Verified account @nickcammarata Oct 15

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had

with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works

already-clean.goodfire.com
View quoted post on X

Thomas Fel @thomas_fel_ Oct 10

The results of @_jacobprince_ heroic effort and crazy logistics of closed-loop monkey experiments 🐒

+27k feature viz, spanning mid-to-high-level visual areas (inferotemporal cortex, superior temporal sulcus...)

Huge amount of results and a beautiful application of interp!🥹

0:05

Jacob Prince Verified account @_jacobprince_ Oct 9

Leading AI models appeared to predict brain activity equally well…

…until we asked them to synthesize images that precisely control neural firing, like these 👇

Across 25 neural sites, closed-loop experiments revealed striking hidden differences in model-brain alignment.

🧵

View quoted post on X