Twitxy Saved

Senthooran Rajamanoharan

@sen_r

London, UK Joined April 2010
43 Following 415 Followers 31 Posts

Latest posts 4

Senthooran Rajamanoharan @sen_r
How does subliminal learning - where fine tuning on meaningless data transfers traits between models - happen?

Some great insights into how this is closely related to learning steering vectors for the same traits, by @camila_blank and @chewing_a_gum
Camila Blank @camila_blank Jun 3
Subliminal learning is when LLMs transmit traits (e.g. loving cats) through seemingly meaningless data. What’s going on?

We find a simple explanation: it's just steering vector distillation.

We explain which traits transfer and why subliminal learning fails across models.
View quoted post on X
1 replies 1 reposts 9 likes 1.7K views View on X
Senthooran Rajamanoharan @sen_r
Today we're releasing Gemma Scope: hundreds of SAEs trained on every layer of Gemma 2 2B and 9B and select layers of 27B! Really excited to see how these SAEs help further research into how language models operate.
Neel Nanda Verified account @NeelNanda5 Jul 31, 2024
Sparse Autoencoders act like a microscope for AI internals. They're a powerful tool for interpretability, but training costs limit research

Announcing Gemma Scope: An open suite of SAEs on every layer & sublayer of Gemma 2 2B & 9B! We hope to enable even more ambitious work
View quoted post on X
2 replies 0 reposts 19 likes 1.2K views View on X
Senthooran Rajamanoharan @sen_r
Happy to see our new paper on JumpReLU SAEs featured on Daily Papers from @huggingface - and looking forward to releasing hundreds of open SAEs trained this way on Gemma 2 soon!
AK Verified account @_akhaliq Jul 22, 2024
Google presents Jumping Ahead

Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders

Sparse autoencoders (SAEs) are a promising unsupervised approach for identifying causally relevant and interpretable linear features in a language model's (LM) activations. To be
View quoted post on X