Pinned post
Matthieu wyart
Verified account This account is verified. Learn more@MatthieuWyart
Theoretical physics × machine learning
Scaling laws | statistical mechanics | learning theory
Prof @ Johns Hopkins & EPFL
Trying to understand why LLMs work
Scaling laws | statistical mechanics | learning theory
Prof @ Johns Hopkins & EPFL
Trying to understand why LLMs work
111 Following 1.3K Followers 58 Posts
Latest posts 5
We’re hiring! PhD students & postdocs. To apply: motivation letter engaging on this topic+ CV, with 2 recommendation letters (PhD) or 3 (postdoc), to matthieuwyart@gmail.com. Applications from women are particularly encouraged. Please share! #PhDposition #postdoc
Our approach: build richer and richer synthetic models of language - simple enough to understand theoretically how deep nets learn them, rich enough to capture meaning, from paraphrases to translations to images. Theory meets linguistics meets high-dimensional data analysis.View quoted post on X
Our approach: build richer and richer synthetic models of language - simple enough to understand theoretically how deep nets learn them, rich enough to capture meaning, from paraphrases to translations to images. Theory meets linguistics meets high-dimensional data analysis.
How can language skills and creativity emerge from examples alone, just by predicting the next token? The structure of data is central. Linguists have studied it most deeply - but real languages are too complex to analyze how networks learn them.View quoted post on X
How can language skills and creativity emerge from examples alone, just by predicting the next token? The structure of data is central. Linguists have studied it most deeply - but real languages are too complex to analyze how networks learn them.
Thrilled that our ERC Synergy grant DeCoR, with Marco Baroni (UPF) and Alessandro Laio (SISSA), is funded. Our goal: decipher how deep networks encode and learn meaning - and lay the ground for a new field: the #PhysicsOfSemantics 🧵View quoted post on X
Thrilled that our ERC Synergy grant DeCoR, with Marco Baroni (UPF) and Alessandro Laio (SISSA), is funded. Our goal: decipher how deep networks encode and learn meaning - and lay the ground for a new field: the #PhysicsOfSemantics 🧵
View quoted post on X