Fabian Fuchs

@FabianFuchsML

Senior Research Scientist @ DeepMind. I work on LLM post-training, focusing on improving training data. Prior to that, I worked on AlphaFold.

Typos are my own.

Oxford, England fabianfuchsml.github.io Joined June 2018

315 Following 2.7K Followers 152 Posts

Latest posts 5

Pinned post

Fabian Fuchs @FabianFuchsML Sep 10, 2021

A year ago I asked: Is there more than Self-Attention and Deep Sets? - and got very insightful answers. 🙏 Now, Ed, Martin and I wrote up our own take on the various neural networks architectures for sets.

Have a look and tell us what you think! :)

➡️fabianfuchsml.github.io/learningonsets/ ☕️

Fabian Fuchs @FabianFuchsML Jul 7, 2020

Both Max-Pooling (e.g. DeepSets) and Self-Attention are permutation invariant/equivariant neural network architectures for set-based problems. I am aware of a couple of variations for both of these. Are there additional, fundamentally different architectures for sets? 🤔
View quoted post on X

Fabian Fuchs @FabianFuchsML May 29

I wrote a blog post trying to understand TrackStar, a gradient-based method tracing LLM predictions to influential training examples.

Mostly: take the main equation, read it from right to left, and poke at the pieces with a small MNIST toy example. 🙂

☕️ fabianfuchsml.github.io/trackstar/

Fabian Fuchs @FabianFuchsML May 4, 2023

Graphs , Sets, Universality

We put more work into this and are presenting it via the ICLR blogpost track (thanks to organisers and reviewers!). Have a read and let us know what you think:

iclr-blogposts.github.io/2023/blog/2023…

better in light mode💡, dark mode🌙 messes with the latex a bit

Petar Veličković Verified account @PetarV_93 Nov 21, 2022

📢 New blog post!
Realising an intricate connection between PNA (@GabriCorso @lukecavabarrett @dom_beaini @pl219_Cambridge) & the seminal work on set representations (Wagstaff @FabianFuchsML @martinengelcke @IngmarPosner @maosbot), Fabian and I join forces to attempt to explain!
View quoted post on X

Fabian Fuchs @FabianFuchsML Nov 21, 2022

I have recently had a range of very insightful conversations with @PetarV_93 about graph neural networks, networks on sets, universality and how ideas have spread in the two communities. This is our write up, feedback welcome as always! :)

➡️fabianfuchsml.github.io/universalgraphs ☕️

Fabian Fuchs @FabianFuchsML Apr 7, 2022

Graph neural networks often have to globally aggregate over all nodes. How we do this can have a significant impact on performance 🎯. After we recently finished a project on this, I wrote a blog post on this topic. Let me know what you think! :)

➡️fabianfuchsml.github.io/equilibriumagg… ☕️