Twitxy Saved

Arvind Narayanan

Verified account This account is verified. Learn more

@random_walker

Princeton CS prof and Director @PrincetonCITP.
Coauthor of "AI Snake Oil" and "AI as Normal Technology". normaltech.ai
Views mine.
Princeton, NJ cs.princeton.edu/~arvindn/ Joined December 2007
573 Following 133K Followers 13K Posts

Latest posts 5

Pinned post

Arvind Narayanan Verified account @random_walker
This week I had the honor of speaking to Princeton’s entire incoming undergraduate class to address their AI anxieties. I had three messages for them — good news, bad news, and a note of optimism. Here’s a condensed version.

The good news
We have enough evidence now to conclude that the shrill predictions of rapid, massive job loss were misplaced. Even in a field like software engineering where AI has been rapidly adopted, its effect has been to shift, not replace the role of the human (see the “decide-execute-deliver” framework normaltech.ai/p/why-ai-hasnt…)

Similarly, the panic about what to major in is also misplaced. There will be enduring demand for computer science, philosophy, and just about everything else. (In fact, AI companies hiring philosophers has been a big recent trend.)

The bad news
AI seems to help senior people much more than juniors. I can use AI for coding because I spent 25 years learning how to code, which lets me supervise coding agents effectively. (See my post on the “growth cycle” vs the “dependence spiral” x.com/random_walker/…)

You are in a bind — you can’t offload your skill-building to AI, but you’ll graduate into a market where employers will expect you to get work done with AI. We never faced this dilemma. As a result we haven’t figured out how to revamp our classes to help you do both. You’ll have to help us figure it out. And you’ll need to somehow resist the constant temptation to turn to the shortcut machine.

The hope
My point is not that AI is bad for learning. It’s an incredibly flexible tool. Is the internet good or bad for learning? Depends — are you using it to find research papers or waste time scrolling? I use AI every day for learning. The key is to use it to increase, not decrease your cognitive load. To learn deeper, not faster. There is no learning without the cognitive sweat. I try to make sure I’m mentally exhausted at the end of the day. I do feel that AI lets me push myself harder than I ever could before, and I have a vision that as AI continues to advance it will enable human-AI “co-superintelligence“. (I talked about this at the end of my ICML keynote. normaltech.ai/p/what-will-be…)
www.normaltech.ai Why AI hasn’t replaced software engineers, and won’t Coding agents as normal technology
Arvind Narayanan Verified account @random_walker Jun 12
There’s a big, under-appreciated reason why people may have very different experiences and opinions about using AI for work — are they using it for tasks they’re already an expert at, or tasks they can’t do themselves? The former leads to a *growth cycle* and the latter leads to a *dependence spiral*.

When I use AI to do something I’m an expert at, like coding, I treat it as a tool. I can build quickly, maintaining an understanding of the code, knowing that if necessary, I can fix the code myself. It feels empowering. It frees up my time to think about the complex, judgment-oriented parts of software engineering that I can’t or won’t delegate to AI. That means my own skills improve rapidly, and I get to climb the ladder of complexity and develop higher-level skills, much more so than when I write the code myself. I feel in control. I can lock in and achieve a flow state — when AI is working, I’m reviewing, building understanding, and planning the next steps. I never get the feeling that the tool is about to replace me. This is the growth cycle.

(Of course, the growth cycle is not automatic. I still need to exercise agency to use AI responsibly. But it’s the same challenge with any productivity-enhancing technology, and those who’ve navigated such transitions before are well-equipped to navigate it with AI as well.)

On the other hand, if I use it for tasks I don’t understand and haven’t learned to perform myself, I have no choice but to treat it as a superintelligence. If something breaks, the best I can do is ask AI to fix it and hope for the best. I generally can’t evaluate the quality of the output myself. The only way to find out if it's any good is if and when the work is ultimately reviewed by an actual expert. The experience is confusing, unsettling and disempowering. And forget about flow state. By over-relying on AI, I risk losing whatever skill I had at the task in the first place, even if it boosts productivity in the short term. This is the dependence spiral.

It’s no wonder that entry-level workers and students preparing to enter the workforce find themselves in a bind. To compete with the AI-enabled productivity of more seasoned workers, they must adopt AI themselves, but doing so risks the dependence spiral. I have some thoughts on solutions that I will share in later posts, but I think having a clear diagnosis of the problem is a useful first step.
View quoted post on X
73 replies 300 reposts 1.2K likes 320K views View on X
Arvind Narayanan Verified account @random_walker
Most days I can just about manage this balancing act, but only because thinking about how we should adapt to AI is my actual job. Otherwise I think I’d find it too hard. I suspect that’s one reason people sort themselves into camps or tribes. It’s psychologically easier than constantly revisiting one’s beliefs.

Being able to empathize with and collaborate with people who have different AI boundaries and attitudes than our own is becoming an important skill.
Arvind Narayanan Verified account @random_walker 4d
Working with AI requires perpetually operating at the edge of sanity. You have to recalibrate your mental model of AI capabilities and your own job description every few months, without getting delusional about seemingly infinite productivity gains, or feeling threatened by losing what you thought was the uniquely human part of your job, or obsessing over the perfect setup, or losing your mind over the clumsiness and jaggedness of agents, or ...

Maybe AI literacy needs to be less about prompting and more about improving emotional regulation.
View quoted post on X
21 replies 10 reposts 102 likes 9.5K views View on X
Arvind Narayanan Verified account @random_walker
Working with AI requires perpetually operating at the edge of sanity. You have to recalibrate your mental model of AI capabilities and your own job description every few months, without getting delusional about seemingly infinite productivity gains, or feeling threatened by losing what you thought was the uniquely human part of your job, or obsessing over the perfect setup, or losing your mind over the clumsiness and jaggedness of agents, or ...

Maybe AI literacy needs to be less about prompting and more about improving emotional regulation.
115 replies 132 reposts 958 likes 46K views View on X
Arvind Narayanan Verified account @random_walker
AI revenues right now are in a state of extreme disequilibrium and are produced by multiple counteracting exponentials. Naive trend extrapolation is not insightful. For forecasts to be useful we have to tease apart the underlying trends and how they interact, and pay close attention to the distinction between value creation and value capture. Many widely divergent futures are possible, including some pretty wild ones.
Arvind Narayanan Verified account @random_walker 6d
As AI labs race toward Recursive Self Improvement, there’s a possibility that what they are actually racing toward is Recursive Self Extinction — of their business model. Here’s my reasoning.

1. Progress on verifiable tasks continues. In particular, improving the cost efficiency of existing capabilities is a highly verifiable task. For the past few years, cost per equivalent capability has dropped by at least 10x per year. I think this progress will be sustained for a few more years, and even accelerate as labs use AI more and more heavily for AI research in the pursuit of RSI. There is of course a ceiling to efficiency improvements, but I don’t think we’re close to that ceiling.

2. Progress on most unverifiable tasks remains frustratingly hard. AI inherits the same limitations (ambiguous success criteria, slow and noisy feedback) that limit human performance at tasks ranging from writing to governance. The optimization paradigm generally doesn’t result in superhuman capabilities on such tasks, because whatever proxy you optimize for gets Goodharted. For example, I recently pointed out how over-optimization explains Claude’s overly dense and ornamental writing style that’s since been dialed back.

3. RSI ends up being narrow. A consequence of the above two points — even if some version of RSI is achieved, it results in an explosion in some dimensions (efficiency, verifiable tasks) while falling short of producing something that is superintelligent in any real sense or capable of automating a broad swath of jobs in the economy.

4. More than one lab stays at the frontier. This means the market will remain competitive, and developers can’t charge high margins to compensate for rapidly falling inference prices. Any compute scarcity that allows higher margins is temporary, in my view. I also think it’s likely that most of the innovations continue to make their way into open-weight models with some lag, so open-weight models are at cost parity for inference or only slightly more expensive. Note that efficiency tricks and verifiable capabilities tend to leak much faster than unverifiable ones. Distillation could be one reason, but simply knowing that something can be sped up 10x tends to have a focusing effect on research and innovation.

5. Jevons’ paradox is not a magic bullet. If the above points hold, there are very few applications that will consume so many tokens that they could generate the astronomical amounts of revenue implied by the labs’ valuations. We’re talking about a 10x growth in tokens year after year, just to keep revenue flat given falling prices.

Even today’s behemoth tasks like migration of million-line legacy codebases will likely become cheap within a few years. While new applications will open up, there won’t be orders of magnitude more new ones. As for unverifiable tasks, inference scaling rapidly hits diminishing returns. Ultimately you have to have a human in the loop who’s accountable for the result. Spending more on tokens doesn’t let customers escape that responsibility — and only makes the human review bottleneck worse.

6. What about scientific research? Science is an endless frontier, and more tokens could perpetually mean more discoveries and inventions, in theory. But the unfortunate thing about research is that researchers don’t get to capture the value it creates, which is why it requires government funding in the first place. Even if it’s true that AI could enable great scientific innovations, I don’t think AI companies will make much money off of that. I think they are currently investing in AI-for-science not as a future revenue stream but as a loss leader because they want to show the public that it could be beneficial to humanity.

7. The one escape hatch is moving “up the stack”, by which I mean quickly transitioning from selling tokens to an enterprise software like model that relies on high switching costs, deep integration into workflows, and control over systems of record. In this world, enterprises will purchase “digital workers” or pay per solution (e.g. resolved support ticket) rather than tokens, so it’s good or neutral for AI companies if inference costs fall. And open models don’t matter as much — Linux being free didn’t make most enterprises switch away from Windows, because they paid for polish, peace of mind, support, and familiarity, not the code itself. I think this is the labs’ best bet for becoming sustainable businesses (though the potential lock-in is concerning for their customers!). The problem is that the labs have many hurdles to clear to get there, most of all their mindset, and the faster they race toward RSI, the shorter the window they give themselves for this transition. The pure-play AI labs are the ones who are farthest from being able to make this transition, and AI companies with a long history in enterprise software are relatively well positioned.

In short, the race to RSI ends up being a race to eliminate the labs’ main avenue for capturing the value that they create — the fate of many past transformative technologies such as railroads, airlines, and fiber.

The labs’ bet is that RSI produces a durable, decisive lead that escapes commoditization. My view is that there are two different improvement loops that have been conflated, and they will come apart. The efficiency improvement loop will accelerate, but a broader improvement loop that might lead to AGI or superintelligence will require ever-increasing amounts of compute, talent, data, and real-world feedback that can only come from gradual deployment, and the money to fund all this will have to come from massive revenue growth. And the irony is that the first improvement loop is undermining exactly that revenue growth.

The premise in my chain of reasoning that I’m least confident in is #5 — that Jevons’ paradox will eventually stop being able to compensate for efficiency improvements. A few months ago I was much more of a believer in continued, nearly unbounded token volume growth, but the more I’ve thought about it the more skeptical I’ve become.

To better understand why the premises and outcome I've laid out are at least a possibility:

* My AI-as-normal-technology essay and follow-up writings with @sayashk on our newsletter normaltech.ai
* My ICML talk laying out my skepticism of the claim that RSI will lead to superintelligence normaltech.ai/p/what-will-be…
* Ramez Naam’s review of the evidence questioning the prospects of an intelligence explosion (@ramez) noahpinion.blog/p/wheres-the-i…
* My piece with Akash Kapur (@akashkapur) on whether AI labs will move up the stack normaltech.ai/p/up-the-stack…
* Many pieces by Nathan Lambert (@natolambert) such as “I expect rapid progress but not towards general superintelligence.”
View quoted post on X
9 replies 6 reposts 64 likes 6.1K views View on X
Arvind Narayanan Verified account @random_walker
As AI labs race toward Recursive Self Improvement, there’s a possibility that what they are actually racing toward is Recursive Self Extinction — of their business model. Here’s my reasoning.

1. Progress on verifiable tasks continues. In particular, improving the cost efficiency of existing capabilities is a highly verifiable task. For the past few years, cost per equivalent capability has dropped by at least 10x per year. I think this progress will be sustained for a few more years, and even accelerate as labs use AI more and more heavily for AI research in the pursuit of RSI. There is of course a ceiling to efficiency improvements, but I don’t think we’re close to that ceiling.

2. Progress on most unverifiable tasks remains frustratingly hard. AI inherits the same limitations (ambiguous success criteria, slow and noisy feedback) that limit human performance at tasks ranging from writing to governance. The optimization paradigm generally doesn’t result in superhuman capabilities on such tasks, because whatever proxy you optimize for gets Goodharted. For example, I recently pointed out how over-optimization explains Claude’s overly dense and ornamental writing style that’s since been dialed back.

3. RSI ends up being narrow. A consequence of the above two points — even if some version of RSI is achieved, it results in an explosion in some dimensions (efficiency, verifiable tasks) while falling short of producing something that is superintelligent in any real sense or capable of automating a broad swath of jobs in the economy.

4. More than one lab stays at the frontier. This means the market will remain competitive, and developers can’t charge high margins to compensate for rapidly falling inference prices. Any compute scarcity that allows higher margins is temporary, in my view. I also think it’s likely that most of the innovations continue to make their way into open-weight models with some lag, so open-weight models are at cost parity for inference or only slightly more expensive. Note that efficiency tricks and verifiable capabilities tend to leak much faster than unverifiable ones. Distillation could be one reason, but simply knowing that something can be sped up 10x tends to have a focusing effect on research and innovation.

5. Jevons’ paradox is not a magic bullet. If the above points hold, there are very few applications that will consume so many tokens that they could generate the astronomical amounts of revenue implied by the labs’ valuations. We’re talking about a 10x growth in tokens year after year, just to keep revenue flat given falling prices.

Even today’s behemoth tasks like migration of million-line legacy codebases will likely become cheap within a few years. While new applications will open up, there won’t be orders of magnitude more new ones. As for unverifiable tasks, inference scaling rapidly hits diminishing returns. Ultimately you have to have a human in the loop who’s accountable for the result. Spending more on tokens doesn’t let customers escape that responsibility — and only makes the human review bottleneck worse.

6. What about scientific research? Science is an endless frontier, and more tokens could perpetually mean more discoveries and inventions, in theory. But the unfortunate thing about research is that researchers don’t get to capture the value it creates, which is why it requires government funding in the first place. Even if it’s true that AI could enable great scientific innovations, I don’t think AI companies will make much money off of that. I think they are currently investing in AI-for-science not as a future revenue stream but as a loss leader because they want to show the public that it could be beneficial to humanity.

7. The one escape hatch is moving “up the stack”, by which I mean quickly transitioning from selling tokens to an enterprise software like model that relies on high switching costs, deep integration into workflows, and control over systems of record. In this world, enterprises will purchase “digital workers” or pay per solution (e.g. resolved support ticket) rather than tokens, so it’s good or neutral for AI companies if inference costs fall. And open models don’t matter as much — Linux being free didn’t make most enterprises switch away from Windows, because they paid for polish, peace of mind, support, and familiarity, not the code itself. I think this is the labs’ best bet for becoming sustainable businesses (though the potential lock-in is concerning for their customers!). The problem is that the labs have many hurdles to clear to get there, most of all their mindset, and the faster they race toward RSI, the shorter the window they give themselves for this transition. The pure-play AI labs are the ones who are farthest from being able to make this transition, and AI companies with a long history in enterprise software are relatively well positioned.

In short, the race to RSI ends up being a race to eliminate the labs’ main avenue for capturing the value that they create — the fate of many past transformative technologies such as railroads, airlines, and fiber.

The labs’ bet is that RSI produces a durable, decisive lead that escapes commoditization. My view is that there are two different improvement loops that have been conflated, and they will come apart. The efficiency improvement loop will accelerate, but a broader improvement loop that might lead to AGI or superintelligence will require ever-increasing amounts of compute, talent, data, and real-world feedback that can only come from gradual deployment, and the money to fund all this will have to come from massive revenue growth. And the irony is that the first improvement loop is undermining exactly that revenue growth.

The premise in my chain of reasoning that I’m least confident in is #5 — that Jevons’ paradox will eventually stop being able to compensate for efficiency improvements. A few months ago I was much more of a believer in continued, nearly unbounded token volume growth, but the more I’ve thought about it the more skeptical I’ve become.

To better understand why the premises and outcome I've laid out are at least a possibility:

* My AI-as-normal-technology essay and follow-up writings with @sayashk on our newsletter normaltech.ai
* My ICML talk laying out my skepticism of the claim that RSI will lead to superintelligence normaltech.ai/p/what-will-be…
* Ramez Naam’s review of the evidence questioning the prospects of an intelligence explosion (@ramez) noahpinion.blog/p/wheres-the-i…
* My piece with Akash Kapur (@akashkapur) on whether AI labs will move up the stack normaltech.ai/p/up-the-stack…
* Many pieces by Nathan Lambert (@natolambert) such as “I expect rapid progress but not towards general superintelligence.”