Patrick Mineault

Excited to release what we’ve been working on at Amaranth Foundation, our latest whitepaper, NeuroAI for AI safety! A detailed, ambitious roadmap for how neuroscience research can help build safer AI systems while accelerating both virtual neuroscience and neurotech. 1/N

People keep telling me my blog posts are too long. I'm thinking I could add narration so you can listen in the gym/on a walk. What appeals to you?

We've been trying to put new neurons in the CNS for ~40 years. One cell therapy for Parkinson's disease is conditionally approved in Japan; a couple more are down the pipe. What's the upper bound on this modality? What's taking so long? And could it be accelerated by AI?

After staring at the activation atlas for a while, I'm less and less convinced about the idea that ANNs have "clean" latents. Nick makes the argument that neurons deep in medium + large ConvNeXt are clean; and they do look cleaner, to my eye, than inceptionv1's neurons at similar stages.

But the initial layers seem more entangled to me. InceptionV1 had clean preferences for edges in its initial layers. ConvNeXt's initial layers look like texture superpositions, or a random rotation of V1/V2 neurons. It's really only at the later stages that we start to see clean, nameable directions.

Is that really surprising? These networks are trained to separate images into different buckets through linear decision boundaries. @klindt_david has a paper deriving a theory of when we should expect this to force the network to separate latent variables. The technical conditions involve the number of dimensions in the representation vs. the number of latents, the objective function, and the shape of the activation functions.

So perhaps we have stumbled into a regime where we have cleaner late-stage representations compared to earlier efforts, but worse early stage representations: where are the Gabors and curve neurons? How can I trust that the features leveraged by the models are robust and generalizable?

already-clean.goodfire.com/microscope/med…

A friend is asking for a good lecture series to learn about neuroscience. Recs?