John Hewitt

This account is verified. Learn more

@johnhewtt

Assistant Prof @columbia CS. Visiting Researcher @ Google DeepMind. PhD from @stanfordnlp. Language x Neural Nets.

New York, NY cs.columbia.edu/~johnhew Joined February 2015

67 Following 8.2K Followers 292 Posts

Latest posts 5

Pinned post

John Hewitt @johnhewtt Oct 20

New from the lab: @LorenaYannnnn finds that reward misspecification during RL causes unmonitorable chain-of-thought. Models talk about the goals in their prompt, even as they learn throughout RL to optimize the misspecified reward. We also propose an intuitive fix; see below.

John Hewitt @johnhewtt 5d

A single direction in LLM activation space, applied as a steering vector with four different norms, can cause the language model to perform a different behavior at each norm.

This and more in a new blog post, Do Not Rely on the Linear Representation Hypothesis. (link below.)

John Hewitt @johnhewtt Oct 26

New blog: separating forms from estimators, or, why steer by difference of means?

Interp often conflates the form (steering, neuron analysis) with the estimator (diff, SGD on log-loss.) When you can learn by gradient descent, you usually just should.

cs.columbia.edu/~johnhew/forms…

John Hewitt @johnhewtt Oct 22

My favorite insight from this work is that monitorability problems are just way easier if you make use of the trajectories and rewards generated during RL. They have so much to tell you about how your model has learned to (mis)behave. Our reward description training does this.

0:14

Tianyi Lorena Yan Verified account @LorenaYannnnn Oct 20

Super excited to share our new paper: Reward misspecification degrades chain-of-thought (CoT) monitorability!

During RL training, when the model is prompted with one task intent (e.g., response ≤ 900 words) but rewarded for a different or even contradictory behavior (e.g., response > 900 words), the CoT keeps reasoning about the task intent while the output violates it.

We also propose a simple mitigation method that requires no prior knowledge of the true reward or the reward hack behavior. 🧵

View quoted post on X

John Hewitt @johnhewtt Oct 6

My lab today at COLM: @GeorgeMorgulis presenting Subliminal Steering, 11:00-1:00 in Franciscan A #78

If you’re studying subliminal learning in LMs, you should be studying George’s variant, which is induced by steering. It just works way more consistently.