Twitxy Saved

Soham De

Verified account This account is verified. Learn more

@sohamde_

Training at OpenAI. Prev: Research Scientist at DeepMind, PhD at the University of Maryland.
London, England sohamde.github.io Joined May 2011
1.2K Following 3.2K Followers 193 Posts

Latest posts 5

Soham De Verified account @sohamde_
Our new paper sheds light on the process of knowledge acquisition in language models, with implications for
- data curricula
- the challenges of learning new knowledge when fine-tuning
- the emergence of hallucinations.

Nicolas did a great job on the project! See his thread👇
Nicolas Zucchet Verified account @NicolasZucchet Mar 31, 2025
Large language models store vast amounts of knowledge, but how exactly do they learn it?

Excited to share my @GoogleDeepMind internship results, which reveal the fascinating dynamics behind factual knowledge acquisition in LLMs!

arxiv.org/abs/2503.21676
View quoted post on X
1 replies 6 reposts 39 likes 5.8K views View on X
Soham De Verified account @sohamde_
It was fun to moderate this discussion with a great group of panelists. Lots of interesting points made on how to approach the next gen of seq modelling architectures. Thanks for the invite @caglarml @orvieto_antonio Razvan and others!
Caglar Gulcehre Verified account @caglarml Jul 26, 2024
The panel discussion at NGSM workshop going on full steam ahead with a great line of panelists moderated by @sohamde_ ...
View quoted post on X
0 replies 0 reposts 13 likes 1.4K views View on X
Soham De Verified account @sohamde_
🔥 Introducing our 9B language model, trained on 2 trillion tokens! 🚀

Based on Griffin (arxiv.org/abs/2402.19427) and delivers:
💪 Powerful performance
⚡️ Lightning-fast inference

Pretrained and instruction-tuned models now available on HF & Kaggle!

Start building today! 🏗️
Samuel L Smith @SamuelMLSmith Jun 12, 2024
RecurrentGemma-9B is out!
kaggle.com/models/google/…
huggingface.co/google/recurre…

- Uses Griffin architecture, combining linear recurrence with local attention
- Downstream evals comparable to Mistral and Gemma
- Faster inference, especially for long sequences or large batch sizes

1/n
View quoted post on X