Aayush Karan

This account is verified. Learn more

@aakaran31

PhD student @Harvard | Algorithmic insights for foundation models | @PDSoros 2024 | Prev @NVIDIA, @GoogleDeepMind, @citsecurities, @apple

Cambridge, MA Joined November 2023

1.5K Following 3K Followers 259 Posts

Latest posts 5

Pinned post

Aayush Karan @aakaran31 Oct 2

SFT is not dead! 🥳

We found a way to make SFT rival current prevailing posttraining methods, often generalizing better and forgetting less than RL and OPSD. 🤯

Following our prior work on reasoning with sampling, we now introduce sampling to the posttraining stack.

1/n

Aayush Karan @aakaran31 Oct 7

Really cool approach to reducing the cost of finetuning with sampling by "learning the sampling policy" itself!!

x.com/haoqik322/stat…

Murray Kang Verified account @haoqik322 Oct 6

Make SFT great again! ✊

Excited to share my weekend project: E2S (Expert-to-Student) Finetuning — an amortized sampling approach for turning off-policy expert demonstrations into more on-policy SFT data.

The problem: vanilla SFT asks the student model to imitate correct but off-policy expert trajectories, causing unnecessary policy shifts and forgetting.

Our idea is to train a model to speak the expert data in the student’s own language.

We formulate this as a constrained generation problem: the expert defines the constraint — what privileged information should be preserved — while the student decides how to generate a more "on-policy" response under that constraint.

The recent Finetuning with Sampling paper targets this distribution with per-example MCMC. But inference-time searching for every example is expensive.

E2S amortizes the search: learn a conditional sampler across examples with GFlowNets, then reuse it to generate more on-policy SFT targets.

On new tasks, E2S gains +6.0 points over MCMC+SFT and +35.2 points over vanilla expert SFT, while preserving performance on prior tasks.

See details in the blog: mk322.github.io/blog/learned-o…

View quoted post on X

Aayush Karan @aakaran31 Apr 20

I'll be at ICLR in Rio this week to present our work on reasoning with sampling! 🇧🇷

Check out my oral presentation on April 23 (Thursday) in the Amphitheater at 3PM, or our poster session that morning!

If you're around and want to chat, please do reach out! :)

Aayush Karan Verified account @aakaran31 Oct 17, 2025

We found a new way to get language models to reason. 🤯

No RL, no training, no verifiers, no prompting. ❌

With better sampling, base models can achieve single-shot reasoning on par with (or better than!) GRPO while avoiding its characteristic loss in generation diversity.

View quoted post on X

Aayush Karan @aakaran31 Apr 10

Great read!! Love the framing that current models are smarter than we think, and it resonates a lot with what we've found at the sampling level, where going beyond next-token samplers to more sophisticated, sequence-level algorithms can also lead to big performance boosts.

There are a lot of gains still on the table if we carefully rethink how we query LLMs to solve tasks, from the level of scaffolding all the way down to sampling.

alex zhang Verified account @a1zhang Apr 10

x.com/i/article/2041…
View quoted post on X

Aayush Karan @aakaran31 Jan 31

Super exciting to see people push on the idea that reasoning can emerge from well-designed sampling!!

Beautiful work! 🔥

Haitham Bou Ammar Verified account @hbouammar Jan 31

We found that much of LLM “reasoning” doesn’t come from RL training; it comes from how you sample the model.

Building on power sampling (Karan & Du 2025), we show you can approximate global reasoning without MCMC, without training, and 10× faster.

🧠 Inference-time intelligence is real.
📝 Blog ↓
medium.com/@haitham.bouam…

View quoted post on X