Elizabeth Barnes (@BethMayBarnes)

Pinned post

METR’s goal: work w frontier developers to get evidence to public on unreleased models’ loss-of-control risks. Asks: 1.transparency on scope, redactions + limitations 2.staff can freely share info w us on in-scope topics 3.rapid publication. They control redactions + access given

METR @METR_Evals Oct 14

If an AI developer was on track to lose control of its internal agents, would anyone outside the company know?

Today, we are sharing METR’s vision for how we would work with AI developers to openly assess and share evidence about this.

View quoted post on X

Elizabeth Barnes @BethMayBarnes 2d

Getting a lot of criticism that's based on obviously incorrect premises about your motivations or basic facts of the situation is subtly corrupting, it trains you to dismiss criticism and makes it pretty hard for people to get through to you on things you really are wrong about

Elizabeth Barnes @BethMayBarnes 3d

I'm really proud of the the team for how we presented the information here in particular, wrangled a lot of complexity to get to something that's (I hope!) relatively readable and clear

METR @METR_Evals 4d

Our review compares three specific claims between the public and nonpublic versions of the report. We also suggest alternative language for each of these claims, informed by our own analysis as explained in our review.

View quoted post on X

Elizabeth Barnes @BethMayBarnes 4d

We do a bunch of ablations, see appendix, but not sure we got to a totally satisfying answer - more people doing their own analysis here seems good. I want people to be able to check things for themselves and not have to just trust METR's analysis! metr.org/blog/2026-11-0…

metr.org Independent Assessment of AI R&D Acceleration at Anthropic in October 2026 METR's high-access assessment of the degree to which AI is accelerating AI research and development at Anthropic.

Scott Alexander Verified account @slatestarcodex 4d

@StefanFSchubert Do you understand why Opus 5.5 seems above trend on this graph but not on epoch.ai/eci?view=graph… ?
View quoted post on X

Elizabeth Barnes @BethMayBarnes Oct 21

The differences here are not huge - 5-6pp of definitely-not-p-hacked effect (ie default METR-written prompt vs original Anthropic prompt), 10pp if you take the biggest gap. (I don't know why the French are the most concerned...) But I was a little surprised the original prompt was the most favorable across a few different variations and metrics. Given how good the models are at author attribution there's a good chance they know who wrote each prompt, which would potentially explain that pattern if it's ~Ant ~sycophancy plus eval awareness. Lots of interesting things to look into here!

Tim Hua 🪶 Verified account @Tim_Hua_ Oct 20

We believe that Anthropic should include "undermining of safety-relevant decision-making" as key risk pathway.

This is partially inspired by Section 2.20 of their risk report, where Anthropic asked Claude for feedback on their report, and received a (imo) sycophantic answer.

View quoted post on X

Elizabeth Barnes

Latest posts 5