Twitxy Saved

Ron Bodkin

Verified account This account is verified. Learn more

@ronbodkin

Founder, Readback Labs: building a comprehension gate for automated AI R&D. Lead author, Comprehension Audits. Ex-Google Responsible AI, Vector Institute
San Francisco Bay Area substack.com/@ronbodkin Joined December 2008
988 Following 9.1K Followers 3K Posts

Latest posts 5

Ron Bodkin Verified account @ronbodkin
All the more reason to slow down US frontier AI for safety. The threat of Chinese competition is receding even as US companies embrace dangerous RSI without even testing for understanding and accountability
George Verified account @odysseus0z 3d
A room of Chinese AI insiders on why the “3–4 months behind” was never about capability: it was distillation lag. Now the rope is being cut. I translated their discussion and checked the claims against public evidence. x.com/i/article/2118…
View quoted post on X
Ron Bodkin Verified account @ronbodkin
METR estimates Anthropic's acceleration from automating AI R&D at 2x. They note "It is possible that automation is close, and leads to very rapid acceleration once human bottlenecks are removed."

However, it is a choice as to whether an organization takes humans out of the loop. Maintaining human understanding and accountability is critical both to pace the frontier and to ensure human experts understand and stand behind safety claims.

To keep humans in the loop, it's important to test whether people still understand what agents are doing.
(1/2)
METR @METR_Evals 3d
This report shares our three subjective conclusions about risk:

1. We believe that, as of Oct 2026, the pace of capability growth on well-specified tasks is likely about 2x what it was from Mar 2024 to Oct 2025. We’re highly uncertain and would not be surprised by 1.3x–3x.
View quoted post on X
6 replies 0 reposts 8 likes 1.1K views View on X
Ron Bodkin Verified account @ronbodkin
I appreciate that Anthropic is considering requiring human review and approval of some AI R&D decisions.

To make that meaningful it's important to test whether the people involved still understand the work deeply:

x.com/ronbodkin/stat…
Theo Bearman Verified account @theobearman 6d
Anthropic updated its S1 this morning. Notable additions re automated AI R&D and human oversight of it:

"[...] our internal estimates and our expectations regarding future models, rather than our current models, indicate that we may imminently meet our Responsible Scaling Policy’s definition of 'dramatically accelerated progress attributable to AI.' We have not designated any of our current models as exceeding this threshold, and we continue to assess the underlying data. We are also evaluating the implications of meeting the AI research and development capability threshold and will release further reports in the future as we set forth in our Responsible Scaling Policy."

"We are also considering additional measures, including updates to our Responsible Scaling Policy, to require human review and approval of certain decisions in our research and development processes as our models perform an increasing share of that work."

"We believe our investments in safety techniques will enable us to manage these risks and responsibly realize substantially more leverage from Claude than we do today in developing AI models. However, we do not anticipate being able to safely use Claude to fully autonomously design and build successive generations of frontier models without meaningful human supervision or understanding, if and when that becomes technically possible. Accordingly, we are considering updates to our Responsible Scaling Policy to keep humans in control of frontier model development at Anthropic."

sec.gov/Archives/edgar…
View quoted post on X
Ron Bodkin Verified account @ronbodkin
I appreciate the addition of supporting human agency in OpenAI's new Model Spec. It says GPT should help people "understand, verify, revise, explain, or take over."

They should measure this: can people explain what an agent did and predict its behavior? Do they catch "tests pass" when no tests ran?

This can be done with a user study and opt-in analysis of real agentic sessions like Codex. Readback Labs is testing these measures in pilots. These measures are an important complement to automated evals.

I also shared this feedback with OpenAI.
Jason Wolfe Verified account @w01fe Oct 29
We’ve updated the Model Spec, our public description of how we want our models to behave. This revision makes clearer what we want from models as they take on more autonomous work in the world and become capable of helping us in new ways. 🧵

openai.com/index/updated-…
View quoted post on X