Twitxy Saved

Ifdita Hasan

@ifdita_hasan

Stanford, CA ifdita-hasan.github.io Joined March 2023
127 Following 270 Followers 65 Posts

Latest posts 3

Pinned post

Ifdita Hasan @ifdita_hasan
As we move from solving competition maths to Erdos problems to discovering new mathematical structures, we scale inference compute accordingly; models think longer, sample alternative pathways, and synthesize those attempts into a refined output. Yet during training, they are trained to think with only sequential compute, where each generation gets rewarded in isolation.

Excited to share our work SPIRAL: an RL framework that trains models to use all primitives of inference compute — sequential, parallel, and aggregative — via end-to-end learning using solely the reward of the final output. We show that this naturally required standard RL to train a model how to aggregate and set RL to train a model to sample parallel traces optimal for downstream aggregation. As a result, the model learns the optimal way to allocate its test-time compute. Our early results show that SPIRAL achieves significantly better performance as all three primitives of inference compute are scaled. Work with @jubayer_hamid (co-lead), @michaelyli_ , @oshaikh13, @yoonholeee, @DorsaSadigh, @chelseabfinn, @noahdgoodman.

Check out our work: arxiv.org/abs/2606.23595
arxiv.org SPIRAL: Learning to Search and Aggregate Language model reasoning can be substantially improved at test time via scaffolds that scale inference compute across different primitives -- sequential reasoning within a trace, independently...
Jubayer Ibn Hamid @jubayer_hamid Jun 23
The most capable reasoning systems in AI scale inference compute along several axes: sequential compute to think longer, parallel compute to sample many independent attempts, and aggregative compute to synthesize prior traces into a new improved one. But during training, we only optimize how models use sequential compute. This creates a fundamental mismatch between how we ultimately deploy these systems and how we train them, leaving much of search and synthesis unoptimized.

We introduce SPIRAL, an RL framework for making all inference-compute primitives end-to-end learnable: models learn to coordinate sequential, parallel, and aggregative reasoning using only the reward of the final output. Work with @ifdita_hasan (co-lead), @michaelyli_ , @oshaikh13 , @yoonholeee , @DorsaSadigh , @chelseabfinn , @noahdgoodman 🧵
View quoted post on X
1 replies 2 reposts 12 likes 2.2K views View on X
Ifdita Hasan @ifdita_hasan
Deploying language models in scientific discovery domains requires extraordinary amounts of test-time compute for search algorithms. An ideal training algorithm should be designed with this goal in mind - that we want agents to learn how to not only exploit but also optimistically explore novel strategies. The agent should learn how to synergistically explore and exploit.

We propose Poly-EPO, a set RL algorithm that explores and discovers diverse reasoning paths. Work with @jubayer_hamid (co-lead), Shreya, @ShirleyYXWu, @HengyuanH, @noahdgoodman, @DorsaSadigh, and @chelseabfinn.
3 replies 22 reposts 110 likes 53K views View on X
Ifdita Hasan @ifdita_hasan
Learning exploration during on-policy RL is critical – especially for test-time compute scaling, generalization, dynamic planning and tasks that explicitly require creativity. However, inducing exploration in policies remains a challenge in systems ranging from simple gameplay to LLMs. In our new work, we train a policy to explore by optimizing for successful and diverse sets of trajectories.

Polychromic Objectives for Reinforcement Learning
📜 Paper: arxiv.org/abs/2509.25424

A wonderful collaboration with @jubayer_hamid (co-lead), @ellenjxu_, advised by @chelseabfinn and @DorsaSadigh🧵