Twitxy Saved

Sambit Ghosh

@sambitghsh

IBM Research, IISc, Trustworthy AI
Bangalore Joined July 2023
135 Following 13 Followers 4 Posts

Latest posts 1

Sambit Ghosh @sambitghsh
Really excited to share this work! We look at a fundamental question in SAE training: How much data do you actually need to train a Sparse Autoencoder (SAE)?
Give it a read and let us know what you think!
Chirag Agarwal Verified account @_cagarwal Oct 21
Exciting work led by @sambitghsh and @MadelynMathai on the diminishing returns of data in Sparse Autoencoders. Standard practice often uses millions of activation tokens, but it’s unclear whether that scale is necessary for different SAE goals. We investigate this systematically by training different SAE architectures on different LLM families with training budgets ranging from 5M to 500M activation tokens.
View quoted post on X