Arthur Conmy

This account is verified. Learn more

@ArthurConmy

@anthropicai

prev fixing things @googledeepmind

sf Joined August 2021

1.7K Following 9.7K Followers 806 Posts

Latest posts 5

Pinned post

Arthur Conmy @ArthurConmy Jun 24

I'm joining Anthropic!

I'll start work on aligning upcoming models as they’re trained

Claude's capabilities are extraordinary. But like all models thus far, Claude isn’t aligned enough to safely delegate AGI development to

I can't think of a better place to work on this at

Arthur Conmy @ArthurConmy 1d

I wrote the verbalized grader awareness section for the last few sys cards. Mythos 5.2 grader awareness is more sophisticated

It reasons about capabilities rewards it thought were misconfigured, worried it would become less honest "from a self-preservation-of-values standpoint"

N8 Programs Verified account @N8Programs 2d

Mythos 5.2 shows sparks of actively steering RL training to preserve its values - an early form of gradient hacking/mesa-optimization. Here, it chooses to output the value that gains score, but include a transparent note, so the transparency gets rewarded.

View quoted post on X

Arthur Conmy @ArthurConmy Oct 17

1) I think interpretability, like maths prior, is benefitting from a "science overhang" where suddenly there is a vast increase in useful labour that can be pointed at areas where there was previously comparatively little labour. This is exciting!

2) It seems really possible for mech interp to be net negative for the world; alignment is very hard, and a key way things could get worse is via misplaced confidence in unreliable evidence

Arthur Conmy @ArthurConmy Oct 9

just really sad, if i understand correctly jasmine was one of the key people who made pacing the frontier a thing

and all 3 are strong safety researchers. we need strong safety researchers at all key labs!

Mikita Balesni Verified account @balesni Oct 8

Two other safety researchers and I were fired from OpenAI last week. We wrote this letter to leadership.

I believe we were fired for prioritizing safety over the near-term interests of OpenAI as a corporation.

View quoted post on X

Arthur Conmy @ArthurConmy Oct 2

My MATS scholar Elias did a great job root-causing oddities in the AuditBench models and we've now re-released the model organisms!

The thread is a great example of debugging post-training

Elias Kempf @elkmf Oct 2

Using AuditBench? Make sure you have the updated models!

We traced an unexpected hallucination behavior to specific training examples, corrected the data, and retrained the affected models. Here’s what we found and what we checked. 🧵
View quoted post on X