Twitxy Saved

William Merrill

@lambdaviking

Assistant Prof @TTIC_Connect | Science of Pretraining @ Ai2

Will irl, TC0 enthusiast
Seattle, WA lambdaviking.com Joined October 2011
706 Following 5.3K Followers 1.7K Posts

Latest posts 5

Pinned post

William Merrill @lambdaviking
My thesis, ๐˜ˆ ๐˜ต๐˜ฉ๐˜ฆ๐˜ฐ๐˜ณ๐˜บ ๐˜ฐ๐˜ง ๐˜ต๐˜ฉ๐˜ฆ ๐˜ค๐˜ฐ๐˜ฎ๐˜ฑ๐˜ถ๐˜ต๐˜ข๐˜ต๐˜ช๐˜ฐ๐˜ฏ๐˜ข๐˜ญ ๐˜ฑ๐˜ฐ๐˜ธ๐˜ฆ๐˜ณ ๐˜ข๐˜ฏ๐˜ฅ ๐˜ญ๐˜ช๐˜ฎ๐˜ช๐˜ต๐˜ข๐˜ต๐˜ช๐˜ฐ๐˜ฏ๐˜ด ๐˜ฐ๐˜ง ๐˜ญ๐˜ข๐˜ฏ๐˜จ๐˜ถ๐˜ข๐˜จ๐˜ฆ ๐˜ฎ๐˜ฐ๐˜ฅ๐˜ฆ๐˜ญ๐˜ช๐˜ฏ๐˜จ ๐˜ข๐˜ณ๐˜ค๐˜ฉ๐˜ช๐˜ต๐˜ฆ๐˜ค๐˜ต๐˜ถ๐˜ณ๐˜ฆ๐˜ด, is now online:
7 replies 51 reposts 401 likes 32K views View on X
William Merrill @lambdaviking
Fun project led by @jowenpetty ! StateBench is benchmark for long-context state tracking abilities that we're already using internally to develop new Olmo models
Ai2 Verified account @allen_ai Nov 3
Can LLMs track state changes over long textsโ€”like an object changing hands or a variable reassignment?

On our new StateBench eval, frontier models including GPT-6 Sol & Claude Opus 5.5 fell to 13% accuracy or lower on the hardest, longest problems. ๐Ÿงต

github.com/allenai/state-โ€ฆ
View quoted post on X
3 replies 5 reposts 64 likes 4.6K views View on X
William Merrill @lambdaviking
Very excited to read this!

at Ai2 weโ€™ve seen CBS can be subtle, and weโ€™ve been begging @SadhikaMalladi to explain the nuances for us
Sadhika Malladi Verified account @SadhikaMalladi Oct 14
Optimizers are usually benchmarked at one batch size, and scaling rules extend them to different batch sizes. We show: (1) frameworks to derive these rules are not as general as they claim to be, and (2) the best optimizer depends on the batch size. batch-size-scaling.com
View quoted post on X
0 replies 2 reposts 28 likes 3.4K views View on X