@vpacela
Excited about this collaboration!View quoted post on X
We share some insights on why linear probes don't generalize under the linear representation hypothesis and expand on why SAEs are still not enough for compositional OOD generalization under superposition/overcompleteness. 👇
I'm in Seoul for ICML! Excited to meet people and present this as a poster at the Compositional Learning Workshop! Saturday, 4 pm, Auditorium.View quoted post on X
Excited about this collaboration!View quoted post on X
We share some insights on why linear probes don't generalize under the linear representation hypothesis and expand on why SAEs are still not enough for compositional OOD generalization under superposition/overcompleteness. 👇
SAEs fail at OOD tasks. Why?
Features in superposition are linearly representable but not linearly accessible. Instead of discarding sparse coding, we embrace the geometry of superposition and use methods equipped to handle the nonlinearity it induces.View quoted post on X
