Twitxy Saved

Francesco Cagnetta

Verified account This account is verified. Learn more

@Fraccagnetta

Theoretical physicist (bow tie included), inherently out of equilibrium. Studying data structure and deep learning at simplexaisafety.com
Trieste, Italy Joined May 2022
538 Following 574 Followers 121 Posts

Latest posts 5

Pinned post

Francesco Cagnetta Verified account @Fraccagnetta
Whatever you make of already-clean.goodfire.com, it has brought compositionality and generalisation back into the spotlight. I’d like to see more rigorous work testing these ideas.

The Random Hierarchy Model is one example. By controlling the data's compositional structure, we recover several key features of deep learning: advantage of depth, scaling laws, linear representations and circuits.

Well-motivated toy models are invaluable: they help us explain how these phenomena fit together and make falsifiable predictions about real data.

Fun fact: @nickcammarata and colleagues’ earlier “Zoom In” work inspired the RHM, back when we were still thinking about cats and dogs.
2 replies 15 reposts 124 likes 4.1K views View on X
Francesco Cagnetta Verified account @Fraccagnetta
Finally someone said it
Vlado Boza Verified account @bozavlado Sep 20
I don't think we should attribute 60k papers submitted to ICLR to AI. It's a natural evolution of bad incentives, and AI is the tool that fulfills them with minimal effort.

When administration and funding agencies count the number of published papers, everybody figures out the "minimal publishable unit" and splits one solid paper into four crappy ones.
When top AI professors demand NeurIPS publication from applying master's students, each student tries to win the publication lottery with something small and insignificant.

Many papers I read in the past year could have been a tweet with an attached GitHub repository of experiments. We got 15+ pages of write-ups about one-line changes (think something like DyT). Or a brutal mumbo-jumbo mixture of techniques to get improvement in the 3rd decimal place (in many disciplines this is called p-hacking, but we kind of ignore that).

Don't blame AI, blame the system.
View quoted post on X
0 replies 0 reposts 19 likes 3.3K views View on X