Pinned post
Yuda Song
@yus167
434 Following 2.2K Followers 407 Posts
Latest posts 5
馃悙
Maybe a good time to announce that I joined @thinkymachines three month ago.View quoted post on X
Very excited about this release. Some highlights:
- pre-trained in less than 3 days
- RL is a single joint run, no need for cumbersome MOPD
- the model is very finetunable, way more than other OSS models
another great work on optimization + data joint study
Muon achieves lower loss than AdamW after pretraining, but does its advantage persist after fine-tuning?
Surprisingly, the answer depends on pretraining data scale: beyond a sufficiently large pretraining data budget, AdamW outperforms Muon after fine-tuning!View quoted post on X
More evidence doesn't necessarily give better answers! Really cool work showing how retrieval can mislead LLMs and how to be more robust.
1/
When LLMs answer questions with retrieved webpages, several plausible answers may appear.
What if one appears much more often and fits much of what the question asks, but still isn't the right answer?
Excited to share our #EMNLP2026 paper on this!
馃搫arxiv.org/abs/2610.27114View quoted post on X
It's easy to teach the model how to succeed when it has the knowledge for a task -- but how do we teach the model *how to fail* when it lacks the capability? 馃
Why does it matter? If we don't teach the model how to fail, it then takes risky actions or hallucinates. 鈿狅笍 (1/n)
Why does it matter? If we don't teach the model how to fail, it then takes risky actions or hallucinates. 鈿狅笍 (1/n)
1/ RL for improving task success has made LLMs remarkably capable. But what happens when the model fails? Turns out, training for graceful failure can be surprisingly tricky!
Very excited to share our new paper: Training Language Models to Fail Gracefully
馃УView quoted post on X
