Twitxy Saved

James Özden 🔸

@JamesOzden

Farm animal welfare @ Coefficient Giving. Prev: founder of socialchangelab.org, doing research on social movements. Donating 10% w/@GivingWhatWeCan
jamesozden.substack.com Joined October 2021
433 Following 2.3K Followers 1.7K Posts

Latest posts 5

James Özden 🔸 @JamesOzden
Paul: "Yeah there's probably a 1 in 3 chance that AI will overthrow the government in the next year"

Dwarkesh: *doesn't flinch and continues on with questions*

My guy you gotta get out the SF bubble and react with some surprise for the rest of us normies
Dwarkesh Patel Verified account @dwarkesh_sp 2d
.@paulfchristiano thinks that on the current trajectory (absent any slowing down), there is a 1 in 3 chance that we could have superintelligence powerful enough to overthrow the government within a year. (He articulates a concrete scenario).

He argues that the labs should pause further algorithmic progress until they can secure their weights and research secrets. Otherwise China could simply steal the superintelligence in the middle of the intelligence explosion, and the labs would no longer be able to slow down even if they saw their models going off the rails.

He also explains how AIs will be trained during the intelligence explosion, and why even a system trained entirely inside a data center could still have the capabilities profile such that humans would have very little chance of competing with it or controlling it.

And he discusses the plan for alignment - basically, making a superintelligence that actually does what we ask the way we mean it. Fixing reward hacks and building better graders will help up till the next few model generations, but he thinks that no single current approach has better than even odds of letting us control superintelligence. The hope is that the AIs figure out the solution to this problem during the intelligence explosion.

Paul is head of the Alignment Research Center. Previously, he was head of AI safety at the US AI Safety Institute. Before that, he led alignment at OpenAI, where he invented RLHF. He’s recently joined OpenAI’s board (though here he is speaking as a private citizen).
View quoted post on X
5 replies 3 reposts 77 likes 5.2K views View on X
James Özden 🔸 @JamesOzden
I never get why editors are allowed to write headlines for articles. This headline couldn't be wrong.

*CO2* stunning will be banned but stunning is still a legal requirement. The industry will have to move to higher-welfare stunning, like argon.
LBC @LBC Oct 26
Stunning of pigs before slaughter to be banned in victory for animal rights campaigners

lbc.co.uk/article/stunni…
View quoted post on X
James Özden 🔸 @JamesOzden
"why ban foie gras? it's not even that bad compared to chicken factory farming"

*proceeds to do nothing about either issue*

🧐
Philip Verified account @Philip_DT Oct 24
@AndyMasley Well you should try banning that in DC first. The scale of animals involved is so many orders of magnitude greater than that of foie gras that you cant even see them on the same chart. A gavaged duck leads a life of absolute luxury next to that of an industrially raised chicken
View quoted post on X