I haven't posted much lately but wanted to share my thoughts on this (all my personal opinions).
Though I don't talk about it much online, I have been aware of and concerned about eventual AI x-risk for over a decade now (like many, first sparked by reading LessWrong). Similar to many, for a time, those risks seemed more theoretical, but I never dismissed that they'd arrive eventually and my worry has steadily grown since it became clear scaling was working (which was around the time of GPT-3 for me).
That said, ever since I started mainlining sci-fi and discovered the extropy list archives as a teenager, I have been a "partisan for the future". A major reason I abandoned my career in tech and chose to pivot to the (at the time, nascent and uncertain) world of AI for bio was because I was motivated to bring about faster progress towards the unimaginably better world I'd dreamed of, including but not limited to developments like those discussed in Machines of Loving Grace. I also think that the upsides of AI have meaningfully outweighed the downsides so far.
I'm including all this to establish that, temperamentally, I'm a technological and market optimist. People who know me know that I think free markets are one of humanity's greatest inventions. I also am very concerned about over-regulation and stagnation due to vetocracy. For example, I think we would be much better off with less strict and bureaucratic nuclear regulation and think it's a tragedy how much value was lost through this. I also support proposals from people like @RuxandraTeslo to accelerate clinical trials and reduce certain clinical regulations (e.g. loosening GMP requirements for Ph1 trials). To date, I have also been opposed to the idea of an "AI pause" and even didn't sign the original pacing the frontier petition because I felt unsure whether the implementation would be good on net.
With all that context, hopefully it's clear that I don't take supporting the proposals described in this essay lightly. But I do support them and think they represent the best attempt I've seen to find a narrow path to a positive AI future. Even as someone who worries about the regulatory ratchet, given the pace of progress and magnitude of the risks, I believe using thorough safety audits to pace deployment is the right approach. On the whole, I think, if implemented, these proposals will not only help prevent potential catastrophes but also avoid situations where a "warning shot" and society's understandable reaction leads to a worse societal/regulatory situation.
Getting back off my soapbox, I want to be clear that I personally remain laser focused on trying to accelerate biotechnological progress and help the benefits of such progress spread as quickly and widely as possible.
Though I don't talk about it much online, I have been aware of and concerned about eventual AI x-risk for over a decade now (like many, first sparked by reading LessWrong). Similar to many, for a time, those risks seemed more theoretical, but I never dismissed that they'd arrive eventually and my worry has steadily grown since it became clear scaling was working (which was around the time of GPT-3 for me).
That said, ever since I started mainlining sci-fi and discovered the extropy list archives as a teenager, I have been a "partisan for the future". A major reason I abandoned my career in tech and chose to pivot to the (at the time, nascent and uncertain) world of AI for bio was because I was motivated to bring about faster progress towards the unimaginably better world I'd dreamed of, including but not limited to developments like those discussed in Machines of Loving Grace. I also think that the upsides of AI have meaningfully outweighed the downsides so far.
I'm including all this to establish that, temperamentally, I'm a technological and market optimist. People who know me know that I think free markets are one of humanity's greatest inventions. I also am very concerned about over-regulation and stagnation due to vetocracy. For example, I think we would be much better off with less strict and bureaucratic nuclear regulation and think it's a tragedy how much value was lost through this. I also support proposals from people like @RuxandraTeslo to accelerate clinical trials and reduce certain clinical regulations (e.g. loosening GMP requirements for Ph1 trials). To date, I have also been opposed to the idea of an "AI pause" and even didn't sign the original pacing the frontier petition because I felt unsure whether the implementation would be good on net.
With all that context, hopefully it's clear that I don't take supporting the proposals described in this essay lightly. But I do support them and think they represent the best attempt I've seen to find a narrow path to a positive AI future. Even as someone who worries about the regulatory ratchet, given the pace of progress and magnitude of the risks, I believe using thorough safety audits to pace deployment is the right approach. On the whole, I think, if implemented, these proposals will not only help prevent potential catastrophes but also avoid situations where a "warning shot" and society's understandable reaction leads to a worse societal/regulatory situation.
Getting back off my soapbox, I want to be clear that I personally remain laser focused on trying to accelerate biotechnological progress and help the benefits of such progress spread as quickly and widely as possible.
We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.View quoted post on X
Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.
You can read the full post here: darioamodei.com/post/we-must-p…
