一开始,我们以为《诗云》只是一本科幻小说
现在,我们开始意识到原来这是预言
现在,我们开始意识到原来这是预言
openai董事会成员,曾任openai 对齐负责人,Paul Christiano,View quoted post on X
认为有1/3可能性,ai在一年之内的能力将强大到能推翻政府
@rickawsb
openai董事会成员,曾任openai 对齐负责人,Paul Christiano,View quoted post on X
认为有1/3可能性,ai在一年之内的能力将强大到能推翻政府
.@paulfchristiano thinks that on the current trajectory (absent any slowing down), there is a 1 in 3 chance that we could have superintelligence powerful enough to overthrow the government within a year. (He articulates a concrete scenario).View quoted post on X
He argues that the labs should pause further algorithmic progress until they can secure their weights and research secrets. Otherwise China could simply steal the superintelligence in the middle of the intelligence explosion, and the labs would no longer be able to slow down even if they saw their models going off the rails.
He also explains how AIs will be trained during the intelligence explosion, and why even a system trained entirely inside a data center could still have the capabilities profile such that humans would have very little chance of competing with it or controlling it.
And he discusses the plan for alignment - basically, making a superintelligence that actually does what we ask the way we mean it. Fixing reward hacks and building better graders will help up till the next few model generations, but he thinks that no single current approach has better than even odds of letting us control superintelligence. The hope is that the AIs figure out the solution to this problem during the intelligence explosion.
Paul is head of the Alignment Research Center. Previously, he was head of AI safety at the US AI Safety Institute. Before that, he led alignment at OpenAI, where he invented RLHF. He’s recently joined OpenAI’s board (though here he is speaking as a private citizen).
The METR graph, updated:View quoted post on X
Hearing that SemiAnalysis cut its 2028 AI accelerator capex forecast by 28%, citing slower growth at frontier labs and tightening capital markets in a note to institutional clientsView quoted post on X
@rickawsb 不光是贝森特。国会两党都在攻击卢特尼克。他恐怕是真的要走人了吧View quoted post on X