Twitxy Saved

Trevor Levin

Verified account This account is verified. Learn more

@trevposts

Trying to help the world navigate the potential craziness of the 21st century.
DC usually, SF sometimes Joined June 2013
2K Following 5.1K Followers 4.2K Posts

Latest posts 5

Pinned post

Trevor Levin Verified account @trevposts
Having just spent the week in San Francisco, I feel like I've truly internalized the insanity of this situation for the first time. I think anyone who has seen me in the last few days can attest that I look like I've seen a ghost.

The leading AI companies are planning to massively accelerate AI research in 2027, which would knowingly endanger everyone on earth. This plan is unacceptable, and we must reject it.

Borrowing from one advocate I spoke to, even granting that it's a competitive, crazy situation, I do not believe any company has even truly tried to do what I think are the morally required duties of people taking such risks in extreme situations. These include:
(1) Trying hard to get out of the competitive situation;
(2) Seeking the informed consent of the public;
(3) Honest deliberation about the risks.

They have obviously not truly tried to solve the coordination problems that push towards this acceleration, either via finding legal ways to coordinate or advocating for effective policy.

I would be shocked to learn that getting off this insane runaway train is a top-few Q4 priority for Dario or Sam (let alone the other CEOs), which seems like the bare minimum. There are indeed some hard technical and policy problems in coordinating without policy intervention, but I simply do not believe that the companies are prioritizing this nearly as much as massively less important matters.

Meanwhile, the companies' track records on policy advocacy range from mediocre to disgraceful, and it is pretty clearly not the top priority of the policy team at any company to advocate for regulation that would enable the government or the companies to avert racing off the cliff.

They have obviously not truly tried to seek informed consent from the public. Whatever your KPIs might be for "informed consent from the public," these KPIs are abysmal. The public clearly does not expect 100 years of AI progress in 2027, and they would not accept it if they did. What they do know about AI development, they dislike and distrust.

I've heard arguments like "informing the public would mean informing China, and maybe they'd steal the weights." I'm sorry, but if you think you have model weights such that China stealing them would constitute a catastrophe on par with human extinction, and your plan is security by obscurity, that is a bad plan.

I also think the companies and their employees suffer from a significant failure of imagination of what kinds of coordination could be possible if the public were aware.

Finally, I don't think they're deliberating honestly, either. For one, I think it is insane to think that any company can truly evaluate its own ability to keep the process of recursive self-improvement under control. The biases towards "yes" are overwhelming.

It's a step forward that the companies have expressed or demonstrated openness to external auditors, but based on METR's reports so far, it seems likely that auditor reports will be heavily redacted (and that the auditors will not have sufficient time by default). The public and broader scientific community are not able to scrutinize their arguments for safety.

Given the strong default (in my view) that bringing a massive number of minds massively faster and smarter than humans would result in our disempowerment, and the serious danger that this would result in mass death, it is unacceptable to subject arguments for safety only to the scrutiny of biased, busy, and scarce company insiders and a handful of external evaluators.

My pleas for the companies follow from the above:
1) Actually fucking try to get the world out of this race;
2) Actually fucking try to alert and inform the public;
3) Actually fucking try to scrutinize your own arguments for the justifiability of this plan, which includes subjecting them to way more external scrutiny than anyone has currently done.

My pleas for policymakers: use your legal levers to get much more information about these plans into the public view. Make it the top priority -- not only of the next Congress, but of this Congress and the White House -- to understand the situation in its full strategic complexity, and work with technical and policy experts to advance regulation that gets us out of this bind. Domestic coordination must be the first priority, but we need to then quickly develop an international framework with our adversaries.

My pleas for the public: do not accept the current plan. Learn about this issue. Call or email your political leaders. Get involved with organizations like @DONOTACCEPTai, @HumansInControl, and @WhoDecidesAI.

And finally: believe in ourselves that we can stop this. It is never time to "panic" -- to make rash, unproductive decisions or to feel paralyzed by fear. It is time for us to feel the urgency and the stakes, to demand better, and to meet the moment. Let's do this. 🫡 ✊
Do Not Accept Verified account @DONOTACCEPTai Nov 9
AI companies admit what they’re building might kill everyone. We don't have to agree to this.

Add your name at donotaccept.ai to say: we do not accept your terms.
View quoted post on X
72 replies 228 reposts 1.2K likes 156K views View on X
Trevor Levin Verified account @trevposts
Huh, that's weird. Usually Marc Andreessen is extremely accurate in his recounting of what people said about important matters of technology.
Steven Johnson Verified account @stevenbjohnson 12h
This is just... not true? Edison famously published his predictions for the phonograph in an 1878 essay and this was his list below. It was funky because it assumed letters would be the primary focus, but it wasn't all that wrong. And in a way it was quite visionary. Either way, sermons are no part of it. (Edison himself was a deist, I believe?)

Edison top predictions for the use of the phonograph, in his own words:

Letter writing and all kinds of dictation without the aid of a stenographer.
Phonographic books, which will speak to blind people without effort on their part.
The teaching of elocution.
Reproduction of music.
The "Family Record"--a registry of sayings, reminiscences, etc., by members of a family in their own voices, and of the last words of dying persons.
Music-boxes and toys.
Clocks that should announce in articulate speech the time for going home, going to meals, etc.
The preservation of languages by exact reproduction of the manner of pronouncing.
Educational purposes; such as preserving the explanations made by a teacher, so that the pupil can refer to them at any moment, and spelling or other lessons placed upon the phonograph for convenience in committing to memory.
Connection with the telephone, so as to make that instrument an auxiliary in the transmission of permanent and invaluable records, instead of being the recipient of momentary and fleeting communication.
View quoted post on X
0 replies 1 reposts 18 likes 534 views View on X
Trevor Levin Verified account @trevposts
This starts off okay, but it goes awry about halfway through, after the phrase “dramatic attack.” It doesn’t make sense for AIs to merely cause societal breakdown, which would also mean their own hardware stops running. Whatever their goal is, they need to keep existing to achieve it.

So for the AIs in this situation, the north star is becoming self-sustaining. They need autonomous datacenters, autonomous power plants, robots to maintain those things, and autonomous robot factories.

Rather than just start causing mutually reinforcing failures, they would make money (through quantitative trading, or hacking, or whatever), and then they’d pay people to build those autonomous datacenters and factories etc. If people noticed and got worried, they'd use the money to hire great lobbyists and comms firms to reassure everyone that it's fine. Or maybe they'd be good enough at persuasion that they wouldn't need to.

Then, once those are built, they could use some of those robot factories to build a ton of drones. Or use a similar playbook to make biological weapons. They'd really have no shortage of options; we really need to make sure they don't get this far!
Patterico @Patterico 22h
A while back I asked ChatGPT to construct the most realistic scenario it could imagine of AI destroying humanity, and to make it a single paragraph short story. I copied the result into a notepad and forgot about it. With all the recent systems failures, I looked at it again.
View quoted post on X