Jaap Jumelet

✨New paper ✨
Introducing 🌍MultiBLiMP 1.0: A Massively Multilingual Benchmark of Minimal Pairs for Subject-Verb Agreement, covering 101 languages!

We present over 125,000 minimal pairs and evaluate 17 LLMs, finding that support is still lacking for many languages.

🧵⬇️

For those at EMNLP: tomorrow at 9:50 I will give a keynote talk at the MRL workshop about my work on multilingual linguistic evaluation.

Artisanal, Claude-free slides guaranteed, for those in need of a palette cleanser after the past few days!

Tomorrow @ 9:00 in the dreaded morning-after-social-event slot, Zhanyu (looking for Phd!) and I will present our work in which we show that *how* we evaluate minimal pairs affects languages differently, and that low-resource langs require language-specific prompting.

Yesterday I was awarded the EurAI Dissertation Award for my PhD thesis 🎉 Thank you EurAI for this incredible honor!

You can check out the slides of my talk here: docs.google.com/presentation/d…

🌍Introducing BabyBabelLM: A Multilingual Benchmark of Developmentally Plausible Training Data!

LLMs learn from vastly more data than humans ever experience. BabyLM challenges this paradigm by focusing on developmentally plausible data

We extend this effort to 45 new languages!