Peter Hase

This account is verified. Learn more

@peterbhase

I work in grantmaking for AI safety and interpretability

Currently: Schmidt Sciences, Stanford
Previously: Anthropic, AI2, Google, Meta, UNC Chapel Hill

New York, NY peterbhase.github.io Joined April 2019

1.2K Following 4.1K Followers 600 Posts

Latest posts 5

Pinned post

Peter Hase @peterbhase Mar 4

Can we train models to have more monitorable CoT?

We introduce Counterfactual Simulation Training to improve CoT faithfulness/monitorability.

CST produces models that admit to reward hacking and deferring too much to Stanford profs (@chrisgpotts told me this is very dangerous)

Peter Hase @peterbhase 5d

It feels like interpretability is finally reaching maturity! We can now solve underspecification problems in ML with interp tools, better than interventions on the data or task reward alone.

Huge congrats to the Goodfire team for making progress here. I have already learned a lot from reading the paper

Thomas Fel Verified account @thomas_fel_ 6d

A science of post-training can be built on interpretability.

Models know when they reward hack, and we see it, yet training ignores it. We propose Mechanistic Shaping: using the model’s own concepts to write training rules.

In vision, DPO & RL, and its effects on monitoring 🧵

View quoted post on X

Peter Hase @peterbhase Nov 4

Happy to share that our paper on beliefs in LLMs (particularly on coherence norms) was published in Artificial Intelligence (the journal)! It just took a couple years :)

doi: doi.org/10.1016/j.arti…

Peter Hase Verified account @peterbhase Jun 6, 2024

New CS+Philosophy paper! 📰

Are language models rational?

This is a key question for *theory of interpretability*. It would be convenient to use the equation “behavior=beliefs+desires” when explaining LLMs, but can we treat LLMs as rational things? Do coherence norms apply?
🧵

View quoted post on X

Peter Hase @peterbhase Nov 2

So tired of Rationalist attempts at argument.

Scott Alexander: I haven’t seen a careful analysis of this amorphous theoretical question, so I’m going to substitute my intuition. Also I asked an AI

Peter Hase @peterbhase Oct 27

Nice paper -- they improve monitorability of reward hacking with a method that doesn't assume prior knowledge of the hacks. This is a pretty common shortcoming of other methods (e.g. inoculation prompting)

Tianyi Lorena Yan Verified account @LorenaYannnnn Oct 20

Super excited to share our new paper: Reward misspecification degrades chain-of-thought (CoT) monitorability!

During RL training, when the model is prompted with one task intent (e.g., response ≤ 900 words) but rewarded for a different or even contradictory behavior (e.g., response > 900 words), the CoT keeps reasoning about the task intent while the output violates it.

We also propose a simple mitigation method that requires no prior knowledge of the true reward or the reward hack behavior. 🧵

View quoted post on X