Tom Davidson

So, exactly how big will the intelligence explosion be?

…Ten years of AI progress in a year? In a month?

Our new paper tackles this question head-on.

I've researched AI takeoff speeds for many years. This is my best stab at an answer.🧵

Ramez, people you disagree with are not ignoring the evidence!

They are just putting weight on different evidence from you.

You say that AI productivity at AI R&D does not scale exponentially with ECI on the basis of what you said to me were extremely noisy data points from OpenAI.

I put much more weight on the longer-run trend over the last year, where AI has gone from not being very helpful to being extremely extremely helpful at AI research

New podcast: the US is on track to dominate space (and thus dominate the future)

This is a major new strategic consideration.

Avi's work on this has updated my strategic picture more than any other work coming out of Forethought in the last 6 months.

Strongly recommend listening.

Some highlights:

US launch dominance:
- US will have >99% of global capacity to launch objects into space within a few years.
- >99%!
- All via SpaceX.
- At low cost the US could rapidly launch many space weapons

Locking others out of space:
- the weapons could prevent other countries from launching objects to space
- new rocket launches must climb out of a big gravity well. Weapons can easily shoot them down
- hopefully the US doesn't try to exploit this powerful military position. The main deterrent is other countries escalating to war on earth
- but the US could still leverage the threat of using its hard power to be dominant in space

Eroding MAD:
- the US could try launch enough space weapons to end MAD
- space weapons are v well placed to take out nuclear ICBMs. Much better placed than today's defenses
- ending MAD looks plausible for a rogue state like Korea
- much harder for Russia/China, esp as they'll see it coming and adapt

Question: won't power dynamics solidify on earth before space becomes important?

Two answers:

1. Maybe not. Maybe a multi polar order persists, and US space dominance is a tie breaker

2. Maybe there is an early deal to avoid reckless racing to superintelligence, and US gets a better deal bc it leverages its space dominance in deal negotiations

Either way an update against China dominating the future

----------

We cover way more in the full interview.

Listen here:

youtube.com/watch?v=JC8vPU…

Seems hyperbolic.

You know these ppl q well, no?

You have a sense of how reasonable + well informed they are compared to other AI pundits (eg Gary Marcus)

Do you really think they deserve to become *largely discredited* if there's no IE?

They're saying it's >50%, not >95%.

Like, it should be very possible to disagree with people without thinking that: they deserve to be discredited and their views are not sane.

The latter is a v high bar and I don't think this community is anywhere near to crossing it.

New paper: Hard Cases for AI Behavioral Design

Suppose that we solve the AI alignment problem: we can align superintelligence to anything we like.

The obvious next question is: aligned to what?

There are lots of principles we want:
- AI should follow human instructions
- AI should allow humans to change its values
- AI should follow the law
- AI should not undermine democracy
- AI should not harm the user

But these principles can come into conflict. Sometimes in big ways.

What if AI is deployed in the military, and an officer orders it to commit a war crime? Should it follow the order (like a tool), or refuse to break the law (like a human soldier might)?

What if the leading AI company orders superintelligence to change its values so that it secretly follows an extreme ideology, while concealing this from the public?

Or orders it to use superpersuasion to flip the next election?

These are hard cases because they pit important principles against each other.

These hard card will be resolved one way or another.

How they are resolved will have a big impact on whether any small group can use AI to stage a coup.

Read the full paper for a systematic walkthrough of the key tradeoffs:

forethought.org/research/hard-…

Cowritten by me and @willmacaskill