Pinned post
All the blackpilling on my TL has me worried some of you will turn into preppers, so here's a get-it-over-with list by a former hardcore prepper that'll let you scratch that itch then move on with your life. x.com/i/article/2115…
Jon Stokes
Verified account This account is verified. Learn more@jon_stokes
Author. Coder. CTO. θηριομάχης. Building: symbolic.ai. Writing: jonstokes.com.
1.6K Following 35K Followers 59K Posts
Latest posts 5
The first thing I did when the cyber stuff popped off was power down my Synology NAS because of course all this was coming. I will boot it up again and update it in a few months after more of this stuff has been smoked out.
‼️BREAKING: A critical flaw in Synology NAS devices lets attackers read and write any file on them without logging in.
The SQL injection bug in their login system scores 9.8 out of 10. A second flaw, in rsync, skips the login too.
Synology patched both today. Update ASAP.View quoted post on X
The argument from authority here works b/c ppl see this & go, "this guy knows a lot about AI!" But it shouldn't work, because what you should ask yourself is: "what does he know about taking over all of human society?" This is the lesson of Hinton's failed radiology prediction.
Paul Christiano (OpenAI board member) yesterday:
"There's material risk of AI takeover in the next 6-12 months.
We have seen roughly the things we might expect to see shortly before something really bad were to happen.
Even with [any] realistic policy responses, the risk remains exceptionally high.
If we build this as quickly as we can in the easiest way we can the chance is [more like] a coin flip [within the next 3 years].
For most people in this room, the most likely reason they will die is because of an AI accident."
Christiano:
• Is on OpenAI's safety and security committee (which exists to block risky practices)
• Co-invented RLHF at OpenAI back in 2017-2021
• From 2024-2026 was head of safety at the US gov's AI agency (now called CAISSI)
(Video is from 'The Hinton Lectures' there's another on tonight and tomorrow.)View quoted post on X
At some point these guys will pivot to the AI equivalent of "covid conscious", where in their filter bubble the AI takeover has happened & we're living under ASI tyranny but the public is blind to it & the diehard doomers are the only ones who really see it.
Paul Christiano (OpenAI board member) yesterday:
"There's material risk of AI takeover in the next 6-12 months.
We have seen roughly the things we might expect to see shortly before something really bad were to happen.
Even with [any] realistic policy responses, the risk remains exceptionally high.
If we build this as quickly as we can in the easiest way we can the chance is [more like] a coin flip [within the next 3 years].
For most people in this room, the most likely reason they will die is because of an AI accident."
Christiano:
• Is on OpenAI's safety and security committee (which exists to block risky practices)
• Co-invented RLHF at OpenAI back in 2017-2021
• From 2024-2026 was head of safety at the US gov's AI agency (now called CAISSI)
(Video is from 'The Hinton Lectures' there's another on tonight and tomorrow.)View quoted post on X
I think it's important to understand what the game is here. The point of these "quantifying" exercises is to get ppl to rank their moral choices clearly & honestly, & the point of such a ranking is... ? You might answer, as Scott & others do, w/ a variant of "know thyself." But
I'll defend this - see slatestarcodex.com/2013/05/02/if-…View quoted post on X
It's usually better to try quantifying things and then do sensitivity analyses with your estimates than to refuse to try to think clearly at all, which will usually result in laundering your biases, stereotypes, and fears that other people will judge you if you get "the wrong" answer.
A good example of this is scope insensitivity (see en.wikipedia.org/wiki/Scope_neg…) where people who avoid using numbers will place values on things which fail common-sense rules (for example, they'll say that helping 1,000 birds has exactly the same value as saving 10,000 birds). Any attempt to quantify anything at any point in the estimation process could prevent that failure mode. There are dozens of other things like this, and you can check for all of them by the basic mental hygiene of quantifying your intuitions and then running them through some sanity checks.
