Twitxy Saved

Jakub Pachocki

Verified account This account is verified. Learn more

@merettm

OpenAI
Joined August 2018
26 Following 97K Followers 54 Posts

Latest posts 5

Jakub Pachocki Verified account @merettm
As AI becomes more capable, I'm focused on questions around the pace and implications of progress, and how they should inform the strategy of OpenAI and across the field. I asked Aleksander to come back to OpenAI to work with me on these topics. I value his judgment and am glad to have him on the team again.
Aleksander Madry Verified account @aleks_madry Oct 19
Personal update: Turns out that what I thought was leaving OpenAI became more of a sabbatical.

When I left a few months ago, I felt that the most important thing to focus on was making AI more useful to people. But we’ve since entered a different regime of AI progress. One in which getting the safety and security challenges of increasingly capable AI right has to become a central priority.

So I’ll be returning to OpenAI, joining @merettm to help chart a path through these challenges.

There’s much work to do, so let’s get going.
View quoted post on X
42 replies 60 reposts 1.3K likes 123K views View on X
Jakub Pachocki Verified account @merettm
❤️
Dario Amodei Verified account @DarioAmodei Sep 12
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.

Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.

You can read the full post here: darioamodei.com/post/we-must-p…
View quoted post on X
82 replies 62 reposts 1.8K likes 102K views View on X
Jakub Pachocki Verified account @merettm
I wrote about the state of AI, why I’m concerned about the next few years, and the choices we need to make to keep the future in humanity’s hands.

An Alien Mind: openai.com/index/an-alien…
openai.com An Alien Mind Jakub Pachocki reflects on increasingly capable AI and the challenge of keeping it aligned. He calls for stronger safeguards and international coordination.
960 replies 2.5K reposts 15K likes 7.7M views View on X
Jakub Pachocki Verified account @merettm
I want to prevent a race into unmonitorability kicked off by confused reporting. The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4.

OpenAI has worked to preserve and utilize chain-of-thought monitoring since our very first reasoning models. We deeply care about this technique, as it can give us a view into how model alignment generalizes from its training distribution. I do think it is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes that I will write about soon. But there are things we can do to strengthen it, and it's a core goal of our current research program.