@LauraRuis
Pinned post
Why do reasoning models struggle to control their CoT? Building on @jcyhc_ai and @tomekkorbak et al’s CoT-Control work, we identify one bottleneck: models default to the same opening tokens.
Ban those openings, and controllability jumps.
The First Few Tokens Matter! 🧵⬇️View quoted post on X
Nice paper on CoT controllability.
They find that chain-of-thought monitoring depends partly on models being unable to control what they write in their reasoning.
They traced much of that limit to the first few tokens after the opening thinking tag. Entropy collapses there, and the model falls into a few common openings.
Small changes at that point restore a lot of control. Prefilling a single token raised controllability substantially. Banning frequent opening tokens and adding in-context examples raised average compliance by 34 to 41 points across models.
These interventions mostly restore control over adding or restyling information. Control over leaving information out responded less consistently.
The two kinds behave like separate capabilities. Across models, scores correlate within each group and mostly negatively between groups.
arxiv.org/abs/2610.40939
Chat with Paper: academy.dair.ai/papers/the-fir…View quoted post on X
Rethinking pre-pretraining (PPT)
Allocating compute to synthetic PPT never outperforms pure natural-language training.
In our compute-matched settings: even when natural language is scarce, repeated epochs over a 100M-token corpus beat every synthetic curriculum. 🧵View quoted post on X
transluce.org Incident Investigations | Transluce We publish notable instances of agent activity on the public internet. 
