Twitxy Saved

Jubayer Ibn Hamid

@jubayer_hamid

PhD in CS at Stanford. Prev: undergrad in maths/physics at Stanford, SR at Google.
Stanford, CA jubayer-ibn-hamid.github.io Joined April 2016
212 Following 1.2K Followers 135 Posts

Latest posts 5

Pinned post

Jubayer Ibn Hamid @jubayer_hamid
The most capable reasoning systems in AI scale inference compute along several axes: sequential compute to think longer, parallel compute to sample many independent attempts, and aggregative compute to synthesize prior traces into a new improved one. But during training, we only optimize how models use sequential compute. This creates a fundamental mismatch between how we ultimately deploy these systems and how we train them, leaving much of search and synthesis unoptimized.

We introduce SPIRAL, an RL framework for making all inference-compute primitives end-to-end learnable: models learn to coordinate sequential, parallel, and aggregative reasoning using only the reward of the final output. Work with @ifdita_hasan (co-lead), @michaelyli_ , @oshaikh13 , @yoonholeee , @DorsaSadigh , @chelseabfinn , @noahdgoodman 🧵
14 replies 91 reposts 438 likes 191K views View on X
Jubayer Ibn Hamid @jubayer_hamid
There’s a certain notion of sampling parallel traces that can "see one another" to collectively be useful. One approach to this is through learning a policy that optimizes for this explicitly. An alternative, readily scalable idea for any policy is to simply do correlated sampling. Interesting work from Michael, Anthony and Kanishk!
Michael Y. Li Verified account @michaelyli_ Jul 2
You're wasting FLOPs when scaling inference compute: by independently sampling parallel attempts, you burn compute rediscovering the same solutions.

Introducing QuasiMoTTo: we scale parallel sampling with correlated samples instead! These samples have higher coverage, are marginally exact draws from the LLM, and can be generated in parallel.

Result: same performance with 25-47% fewer samples in test-time scaling + 50% fewer training steps in RL!

In our new paper, we explore the design space of correlated samplers. Work with co-authors @probablynotaz9 (co-lead), @gandhikanishk, @noahdgoodman, and Emily Fox!
View quoted post on X
0 replies 1 reposts 19 likes 2.3K views View on X