Pinned post
We just discovered almost a million public URLs that OpenAI’s agents left behind when hacking Hugging Face, leaking credentials and attack details that could have allowed anyone who found them to compromise the company. 🧵
Jeffrey Ladish
Verified account This account is verified. Learn more@JeffLadish
Applying the security mindset to everything @PalisadeAI
1.5K Following 24K Followers 14K Posts
Latest posts 5
Imagine if a nuclear fusion company had a working reactor & >$20B in annual revenue. They believed that their research would enable them to create a small, more portable reactor in the next year, but also that the reactor could be easily turned into a gigaton nuclear bomb. Independent scientists were divided about whether they could make that new type of reactor in the next year, but generally agreed that if they could, a bomb wouldn't be much harder, and that would be extremely dangerous.
I'm not sure the people leading AI companies, including technical staff, are correct about AI speedup & imminent RSI capabilities. But the fact that they believe it is totally enough justification to stop them!
Fully closing the RSI loop would be much more dangerous than developing a portable gigaton nuclear weapon, even though that's ~200x the yield of the biggest nuclear bomb ever tested.
I'm not sure the people leading AI companies, including technical staff, are correct about AI speedup & imminent RSI capabilities. But the fact that they believe it is totally enough justification to stop them!
Fully closing the RSI loop would be much more dangerous than developing a portable gigaton nuclear weapon, even though that's ~200x the yield of the biggest nuclear bomb ever tested.
“Anthropic probably wouldn’t have been able to speed up its research this much by hiring any number of people” - @ajeya_cotra
Very helpful and evocative description: “When you’re picturing a 10x uplift, you should not be imagining that every researcher is now a team of ten researchers — as anyone who has ever done anything knows, a team of ten does not get ten times as much valuable work done per hour as one person. Instead, it’s as if every researcher at Anthropic is in a time warp bubble where they experience ten days for every day that passes in the outside world. As if AI lets them think, read, speak, type, and do everything else ten times faster. Anthropic probably wouldn’t have been able to speed up its research this much by hiring any number of people”View quoted post on X
Look at this graph. Notice how it’s an exponential *on a log scale*. I pray and hope there will be strongly diminishing returns before long in this process. But leaders at METR and Anthropic both seem to think there might not be. That’s fucking terrifying.
As AI improves, uplift will get even higher. If it overcomes diminishing returns and bottlenecks, a naive extrapolation implies we would reach ASI within a year. The authors say they found no clear bottlenecks or sharp diminishing returns, though they are very uncertain.View quoted post on X
