Twitxy Saved

David Chanin

@chanindav

Joined January 2011
251 Following 223 Followers 48 Posts

Latest posts 5

David Chanin @chanindav
I've been curious how much we can quantize SAE training so we can train huge SAEs on a single GPU. I had Silico evaluate how much we can quantize SAE training before results degrade. 馃У
Goodfire @GoodfireAI Aug 4
Silico, the platform for ambitious AI research, is publicly available today.

AI is advancing fast. The tools to understand it need to advance even faster. Silico lets you interpret and train your models at frontier scale.

Learn more + get access 馃У
View quoted post on X
1 replies 2 reposts 29 likes 1.8K views View on X
David Chanin @chanindav
I let Claude autonomously do research on improving SAEs, and was extremely impressed with the result. The final SAE architecture outperforms all tested architectures on SynthSAEBench-16k, even matching logistic regression probe performance.
4 replies 7 reposts 101 likes 4.8K views View on X
David Chanin @chanindav
SAEs fail even when the Linear Representation Hypothesis holds perfectly.

We built SynthSAEBench: large-scale synthetic data with 16k ground-truth features, correlation, hierarchy, and superposition. We trained 5 SAE architectures on it.

None achieve perfect feature recovery.
5 replies 26 reposts 213 likes 10K views View on X