Katrin Franke

1/9 How does the brain encode what we see? 👁️

We introduce "dual feature selectivity" – many 🐒 🐭 cortical neurons encode a continuum b/w activating & suppressing features

Blog✨enigma-brain.github.io/dual-feature-s…

🙏 co-lead Nikos Karantzas & @AToliasLab @naturecomputes @sinzlab & more

Very excited that our paper is now out at @arxiv🎉

As a neuroscientist🧠, I wand to understand how (single) neurons encode the visual world. But how to do this in an automated yet interpretable way when we record thousands of neurons to complex natural stimuli?

Here, we use vision language models, specifically @GeminiApp, to convert neural selectivity in monkey visual cortex into language, i.e. a semantic hypothesis, which we verify with text-to-image generation!

Check out the below post and this beautiful homepage by @vedanglad: enigma-brain.github.io/letting-the-ne…

Great team effort with @AToliasLab @naturecomputes @SuryaGanguli @TamarRottShaham Nikos Karantzas & star first author @vedanglad 😍

This work was done at @StanfordMed and funded through The ENIGMA Project by @jamesfickel 🙏

Super excited to see OmniMouse 🐭 released 🎉

A single model of mouse visual cortex — queryable for many different questions:

• Predicting neural activity from video 📹 👉 🧠
• Predicting neural activity in one population from activity in another 🧠 👉 🧠
• Forecasting neural activity forward in time, given video, past activity, or both 📹 &🧠 👉 🧠
• Decoding behavior (gaze, pupil size, running speed) from neural activity 🧠 👉 🐁
• Predicting neural responses conditioned on behavior 📹 & 🐁 👉 🧠

This kind of multi-modal & multi-task flexibility is an exciting approach for systems neuroscience, enabling systematic in silico exploration of hypotheses about single neuron and population coding

The dataset comprises >2 million neurons across 78 mice and 328 sessions of mouse visual cortex, with naturalistic and parametric stimuli alongside behavior. Both the dataset and the pretrained models are available on @huggingface 🙌

Congratulations to @KonstantinWille @pollytur1
@alexrgil14, and the wider team across the @AToliasLab, @alxecker, and @sinzlab labs and the Enigma Project /w @naturecomputes. Paper, code, data, and models in the original post 👇

Every experimental neuroscientist knows the feeling: you have a hypothesis, but running the experiment takes months👩‍🔬

In our new preprint @biorxiv_neursci , we present an openly available functional ''digital twin'' of the retinal input to the mouse superior colliculus that lets you test hypotheses in the model first — try it out yourself using the link below 🧠

We combined chronic two-photon imaging of >200k retinal ganglion cell axonal boutons in the mouse superior colliculus (SC) with deep dynamic models that predict neural responses to parametric light stimuli and natural movies.

Key findings⚡️

▸ Retinal inputs to the SC form functionally distinct, laminar-organized response types, identified via Gaussian mixture model clustering

▸ The functional diversity of retinal output matches that of retinal input to the SC. We show this by aligning our dataset with a retinal reference dataset using a variational autoencoder with adversarial training

▸ Our deep dynamic digital twin learns stimulus–response transformations and generalizes to stimuli it was never trained on, including parametric stimuli used for cell type identification

The model functions as a virtual lab bench: feed in any stimulus you're curious about and generate predicted neural responses. As a proof of concept, we fed in a looming stimulus — known to trigger defensive behavior in mice — and identified putative response types selective for this stimulus

Try it in our Colab notebook with your own stimulus and see what the model predicts

📄 Preprint: biorxiv.org/content/10.648…💻

Colab: colab.research.google.com/drive/1k9411tL…📂

Code: github.com/yongrong-qiu/r…

Huge thanks to an incredible cross-institutional team spanning @StanfordMed , @uktuebingen , @uniGoettingen , @bcm_neurosci & many more

@YongrongQ, @lisa_schmors, Na Zhou, Mels Akhmetali, Dominic Gonschorek, Cameron Smith, Anton Sumser, Marie Vallens, @crcadwell, Fabrizio Gabbiani, Maximilian Joesch, @AToliasLab, Philipp Berens, Thomas Euler, @sinzlab, @viajake 🙌

New paper out in @NeuroCellPress 🎉

What determines contextual modulation in primary visual cortex (V1)?

The key result ⚡ V1 neurons are facilitated by surrounds that complete their optimal center feature according to natural scene statistics, and suppressed by surrounds that disrupt it — a principle explained by hierarchical Bayesian inference and conserved across mouse and macaque.

These results converge with Deveau et al. in @NeuroCellPress (cell.com/neuron/fulltex…) from the lab of @HistedLab, who show that recurrent circuits in V1 filter temporal input sequences to selectively boost natural dynamics, and Lange et al. in @ScienceMagazine (science.org/doi/10.1126/sc…) from the lab of @haefnerlab, who show that perceptual learning increases population redundancy as predicted by generative inference. A consistent picture is emerging: (early) visual cortex actively infers the statistical structure of the natural world.

Amazing collaboration with @AToliasLab @haefnerlab @sinzlab Antolik Lab & many more — led by @jiakunfu, with co-authors @suhas_shrinivas & @LuchinoBaroni & many more

The paper is open-access and available here: doi.org/10.1016/j.neur…

More detailed approach:

We trained CNN digital twins on large-scale two-photon recordings from mouse V1 and used them to synthesize, for each neuron individually, the surrounds that most strongly facilitate or suppress its response to its optimal center stimulus. Closed-loop in vivo inception loop experiments confirmed the predictions.

Key qualitative finding:
Surrounds that complete the optimal center feature under natural scene statistics → facilitation
Surrounds that disrupt it → suppression

We verified this with an independent generative diffusion model (blind to our CNN): statistically likely continuations of the optimal center feature were significantly more similar to facilitatory surrounds in V1 representational space. The same principle holds in macaque V1 despite major differences in receptive field organization.

We formalize these results in a hierarchical Bayesian inference model — V1 neurons represent posterior beliefs about local features, with feedback from higher areas encoding global scene structure — and find like-to-like excitatory connectivity in the MICrONS dataset as a candidate circuit mechanism.