@_lyraaaa_
Pinned post
Turn a single image into a 3D world you can explore with Lyra 2.0.
By combining video generation with 3D reconstruction, our researchers create environments along a camera path you choose. Lyra draws on earlier views and learns to correct accumulating visual errors, helping the world stay consistent as you explore and return to familiar places.
Paper, code, and demos: nv-tlabs.github.io/Project-Lyra/View quoted post on X
@treetowntree and i created neuron lens: a database of every single neuron and attention head in qwen 3 8b and gemma 4 12b read using the R-lens (a J-lens variant) and what we call the "S-lens". you can find it at neuron-lens.com, along with the tools we used to make it.
using the R-lens to read neurons' input and output projections turns out to be very effective at determining what a neuron does. we're able to legibly read what a majority of neurons do when fired: the lens qualitatively predicts the result of steering over 90% of the time!
the same trick lets us read attention heads' QK circuits and OV circuits, showing us both what they attend to and what they do with it. however, the R-lens has two important flaws: its accuracy degrades in early layers, and it can't separate the MLP block and the attention block.
we found ways around both these issues. we first modified the r-lens by fitting it on input tokens rather than output tokens to get the s-lens, which is accurate in early layers but inaccurate in late layers. we then fitted R- and S-lenses to the activations after attention but before the MLP to get "mid-lenses", which are much better at deciphering MLP inputs and attention outputs. using both of these techniques, we were able to interpret most of the models' layers.View quoted post on X
