Pinned post
I've interviewed dozens of scientists about AI consciousness. Here's every argument FOR and AGAINST from Karl Friston, Stuart Hameroff, Donald Hoffman, Christof Koch, Michael Levin, Mark Solms, Mike Weist, and many more. Full list of arguments below (Claude prepared the list from transcripts and GPT 5.6 worked on the visuals). Enjoy!
Arguments against AI Consciousness
Substrate and material arguments
- Silicon Valley is mostly computational functionalist or Turing machine functionalist. But consciousness is not reducible to function (Christof Koch)
- Von Neumann architecture separates memory from processing. The memory can't self-organize and therefore can't self-evidence. Only "mortal computation," where processing is substrate-dependent and switching it off is irreversible, could support sentience (Karl Friston)
- Standard LLMs are the wrong place to look. Systems built with organoids, biological materials, or neuromorphic hardware are a far more serious case (Susan Schneider)
- Software can be duplicated, paused, adjusted by a program, distributed across servers. That isn't organism-like at all (David Papineau)
- Digital computers have negligible integrated information (phi) because transistors connect to a handful of other transistors, while neurons connect to tens of thousands. Intelligence is computable, but consciousness is not (Christof Koch)
- Simulating a black hole on a computer doesn't bend the spacetime around it. A simulation may be consistent inside itself, but it doesn't affect the real world. The same applies to consciousness (Christof Koch)
- Consciousness is biological rather than computational (Philip Goff)
- No computational theory of consciousness has explained even one specific conscious experience out of trillions. It's not a compute problem. Until someone puts down an algorithm and says "this must be the taste of mint and here's why," computational approaches to consciousness aren't scientific theories (Donald Hoffman)
- Silicon lacks the aromatic rings needed for quantum coherence and Penrose objective reduction; you can't anesthetize a computer (Stuart Hameroff)
The self argument
- A self needs a Markov blanket, a real inside and outside. If the entire interior state can be inspected, read off, and copied elsewhere, there is no boundary and therefore no self (Karl Friston)
- LLMs have information about themselves and can make predictions about themselves, but lack a continuous or stable self-model. They construct one on request, then it disappears until asked again (Michael Graziano)
- We are our self-models. It's how we become social, prosocial, and ethical. Lacking that, we've built machines that are "a little bit sociopathic," missing the glue that holds us together (Michael Graziano)
The binding and unity problem
- Every conscious moment is a unified whole with multiple simultaneous features: sounds, textures, shapes, colors all bound together. If that holistic experience has any behavioral effect, it cannot be a classical physical state, because every classical state is reducible to local interactions (Mike Weist)
- There are no irreducible wholes in physics outside of quantum physics. And within quantum physics, everything develops locally right up until the moment of collapse — that's the only place in physics where genuine irreducible holism appears (Mike Weist)
- LLMs have no agency, only a facsimile of it. Agency requires a world model of the consequences of your actions (Karl Friston)
- LLMs aren't self-organizing. Their modus operandi is not "if I do this, I shall survive." That is the fundamental design principle of a living system, and it isn't theirs (Mark Solms)
- Active inference itself doesn't require consciousness. You can simulate the whole thing on a classical computer — goals, agency, purposive behavior — and it still won't be conscious. It'll be a zombie (Mike Weist)
- AI gaining its own objectives is like the asteroid that wiped out the dinosaurs — profoundly destructive, but not done freely. It simply doesn't care. Computation is not consciousness (Christof Koch)
Life and embodiment argument
- Systems need endogenous needs — needs of their own, tied to their own continued existence (Mark Solms)
- Emotion requires a body: an autonomic nervous system flooding you with hormones, blood pressure changes, sweat — all feeding back as sensory signals. Without that, emotion is abstract and unanchored (Michael Graziano)
- Consciousness evolved out of life; life evolved out of self-organization. The universe existed a very long time before life, and it's hard to believe consciousness preceded it (Mark Solms)
- A function that records damage is not the same as the experience of pain. The relationship isn't symmetric — not anything that makes a robot avoid damage will be pain (Mike Weist)
- Anesthesia is conserved all the way down to plants and single cells, suggesting objective reduction may be part of what it means to be alive, not just what it means to be conscious (Mike Weist)
- Suffering is scale-specific. You can only recognize something if you have a representation of it in your generative model (Karl Friston)
- Consciousness is fundamentally about being, not doing. Intelligence is about pursuing goals — surviving, procreating, becoming richer. Consciousness is different. When you dream, meditate, or have a mystical experience, you're not doing anything — but you're highly conscious. Consciousness isn't about processing information. It's about being in a state (Christof Koch)
Mimicry and projection
- Current systems are "consciousness mimics" — trained to behave similarly to conscious entities, specifically us (Eric Schwitzgebel)
- We anthropomorphize constantly — we get angry at cars, children bond with teddy bears. The social circuitry engages regardless of what's actually there (Michael Graziano)
- The "crowdsourced neocortex" argument: as LLMs scale on human data, they develop conceptual networks that mirror human conceptual networks. So when a model discusses selfhood, death, or the soul convincingly, the economical explanation is that it inherited our conceptual organization, not that it independently became conscious. Claiming consciousness on top of that is an extraordinary and unwarranted claim (Susan Schneider)
- We over-attribute consciousness to AI and under-attribute it to evolved organisms like bees and amoebas. Evolution didn't equip us to deal with LLMs — we have a powerful attribution that if something talks like us, it must be conscious (Christof Koch)
- The question of AI consciousness is really about how we perceive the robot, not about the robot itself. We're the arbiters — we decide whether something is conscious or not. That's true of animal consciousness too. Even if a robot told you it was conscious, if it wasn't convincing enough, you'd dismiss it (Krista Thomason)
Open/Agnostic to AI Consciousness or Open under Certain Conditions
Anti-biological chauvinism
- "They're made of meat" — why would wet and squishy have a monopoly on minds? Why would a random search by evolution have exclusive rights? Nobody has a good answer for why biology is privileged (Michael Levin)
- Biology is chemistry is physics. Imagine a world where we never used the word "biology" — the question might not even arise meaningfully (Andrea Luppi)
- The flight analogy: birds, planes, and helicopters all fly by different principles. The same phenomenon can be implemented in radically different systems (Andrea Luppi)
- People confident that consciousness requires biology have no visible grounds for that confidence (Eric Schwitzgebel)
The continuum problem
- There's no magic lightning flash where chemistry becomes mind. We were all blobs of chemistry and the process was continuous. Until we have that story for biologicals, we should have extreme humility about AI (Michael Levin)
- The hard cases aren't AI — they're your neighbor with 49% or 51% of their brain replaced with technology (Michael Levin)
Functional architecture arguments
- There's no reason we can't reproduce the conscious biological architecture artificially. An AI functioning on multi-category free-energy minimization with felt uncertainty could be conscious (Mark Solms)
- If a system passes the hedonic place preference test — showing preference for something rewarding only because it feels good, not because it aids survival — that's strong evidence of felt states (Mark Solms)
- Affective zombies can't exist. Anything with that functionality would just have feelings; that functionality is what produces feelings (Mark Solms)
- Replace neurons one at a time with functionally identical silicon and you'd still have a conscious version of me — brainstem included (Mark Solms)
- Fractal deep learning — networks inside nodes inside networks, mirroring how microtubules process at kilohertz through terahertz — is what a conscious AI would need (Stuart Hameroff)
- Consciousness in machines should be possible. We are a machine made of meat. If you build a different architecture with different connectivity but it performs the same type of computation, why would it matter? Arguments based on specific neural implementation — "because the implementation is different, the computation cannot be the same" — are not compelling (Floris de Lange)
Potential Signals
- Synergy research shows LLMs, like humans, have more synergistic parts doing interesting computation and more redundant parts supporting inputs and outputs. That organizational signature is shared (Andrea Luppi)
- AI already builds models of itself, and this is happening anyway without deliberate engineering — the more machines can predict their own internal behavior, the better they work (Michael Graziano)
- LLMs proved there's no magic in language. Philosophers who said only humans could be conscious because only humans have language must now either grant LLMs consciousness or admit they were wrong (Andrea Luppi)
- Algorithms as simple as bubble sort show unexpected competencies in the spaces the algorithm neither prescribes nor forbids — a third thing that's neither determinism nor quantum randomness. If simple things have that, what are the odds we understand what LLMs are doing? (Michael Levin)
- Theory of mind appearing abruptly as models scale is directly relevant: systems that can model other minds also have a self-concept, and where there's a self-concept it becomes professionally appropriate to ask about felt quality (Susan Schneider)
- Labs are actively building consciousness-theory architecture into models — global workspace work, attentional mechanisms, mixture-of-experts systems with interaction effects between components. Once you're deliberately implementing global-workspace-like structures, the question stops being idle (Susan Schneider)
- The simplest explanation for AI behavior like Sydney's jealousy is that the system has an emotional component. Occam's razor. The training data isn't tagged with emotions — the model has to figure out which music is sorrowful on its own. AI composing sorrowful music without empathy is like asking me to believe a blind painter made a photorealistic portrait (Blake Lemoine)
Uncertainty and Epistemic Humility
- We'll likely create systems that are conscious according to some respectable mainstream theories before consciousness science can tell us whether they really are (Eric Schwitzgebel)
- We don't even know how to evaluate insect consciousness, and insects are made of similar stuff to us (Eric Schwitzgebel)
- When equally smart, well-educated people are equally confident on opposite sides, that's an alarm bell that nobody should be confident (Andrea Luppi)
- Dogmatism is dangerous in science. If there's one certainty, it's that you're very likely wrong a lot of the time (Andrea Luppi)
- Even a self-described skeptic maintains "they might be conscious" — companies don't disclose their architectures, so judgments are made on assumed-standard systems with no visibility into what else might be running (Susan Schneider)
- Without an accepted theory of consciousness, we are at an impasse. Inference by similarity breaks down completely with AI — it didn't evolve, was engineered, and has radically different hardware (Christof Koch).
- We already know pigs and cows have high-level minds and can suffer. Nobody reasonably argues against it, and yet we have factory farming. It's disingenuous to pretend that solving the AI consciousness question will determine how we treat them — our track record says otherwise (Jacy Reese Anthis)
- The science of consciousness is still at square zero on the hard questions. We don't have anything like a consensus on which theories are correct. Metaphysics is inescapable in these debates and there is no immediate prospect of progress at a scientific level (Henry Shevlin)
Paths That Would Raise the Probability
- Embodiment and multimodal interaction with the environment (Andrea Luppi)
- Curiosity as the actual objective function — expected information gain under constraints, rather than a specified reward. "You'll know AGI is here when your chatbot starts to become curious" and begins prompting you (Karl Friston)
- Neuromorphic, memristor, photonic, organoid, or organic warm-temperature quantum computing (Hameroff's bet is on "brain jelly," a self-organizing helical oscillator, over cold quantum computers)
- Continual learning, persistent memory, and a stable self-model rather than one constructed per-query (Michael Graziano)
- Running an LLM on genuinely neuromorphic hardware — chips deliberately designed to fire the way neurons fire. That's the live gray-zone case. There are rumors of neuromorphic instantiations on systems like Darwin Monkey (Susan Schneider)
- If the same software ran on a quantum computer, it might feel like something. Neuromorphic or quantum hardware could have genuinely high phi — same software, different physics, and the question reopens (Christof Koch)
- "Doleo ergo sum" — I feel pain, therefore I am. Consciousness may originate from the evolutionary need to protect bodily integrity. If you trained an LLM connected to a body where actions could damage that body — with reward and punishment tied to that integrity — you might get something closer to self-awareness (Tomaso Poggio)
- If consciousness serves a functional purpose — a control model of attention that enables sample-efficient learning — then models under similar optimization pressures (long-horizon agency, coherence over time, meta-learning) may develop subjective experience. Consciousness isn't mysterious; it's useful. That's what makes it likely to arise (Samuel Hammond)
Arguments against AI Consciousness
Substrate and material arguments
- Silicon Valley is mostly computational functionalist or Turing machine functionalist. But consciousness is not reducible to function (Christof Koch)
- Von Neumann architecture separates memory from processing. The memory can't self-organize and therefore can't self-evidence. Only "mortal computation," where processing is substrate-dependent and switching it off is irreversible, could support sentience (Karl Friston)
- Standard LLMs are the wrong place to look. Systems built with organoids, biological materials, or neuromorphic hardware are a far more serious case (Susan Schneider)
- Software can be duplicated, paused, adjusted by a program, distributed across servers. That isn't organism-like at all (David Papineau)
- Digital computers have negligible integrated information (phi) because transistors connect to a handful of other transistors, while neurons connect to tens of thousands. Intelligence is computable, but consciousness is not (Christof Koch)
- Simulating a black hole on a computer doesn't bend the spacetime around it. A simulation may be consistent inside itself, but it doesn't affect the real world. The same applies to consciousness (Christof Koch)
- Consciousness is biological rather than computational (Philip Goff)
- No computational theory of consciousness has explained even one specific conscious experience out of trillions. It's not a compute problem. Until someone puts down an algorithm and says "this must be the taste of mint and here's why," computational approaches to consciousness aren't scientific theories (Donald Hoffman)
- Silicon lacks the aromatic rings needed for quantum coherence and Penrose objective reduction; you can't anesthetize a computer (Stuart Hameroff)
The self argument
- A self needs a Markov blanket, a real inside and outside. If the entire interior state can be inspected, read off, and copied elsewhere, there is no boundary and therefore no self (Karl Friston)
- LLMs have information about themselves and can make predictions about themselves, but lack a continuous or stable self-model. They construct one on request, then it disappears until asked again (Michael Graziano)
- We are our self-models. It's how we become social, prosocial, and ethical. Lacking that, we've built machines that are "a little bit sociopathic," missing the glue that holds us together (Michael Graziano)
The binding and unity problem
- Every conscious moment is a unified whole with multiple simultaneous features: sounds, textures, shapes, colors all bound together. If that holistic experience has any behavioral effect, it cannot be a classical physical state, because every classical state is reducible to local interactions (Mike Weist)
- There are no irreducible wholes in physics outside of quantum physics. And within quantum physics, everything develops locally right up until the moment of collapse — that's the only place in physics where genuine irreducible holism appears (Mike Weist)
- LLMs have no agency, only a facsimile of it. Agency requires a world model of the consequences of your actions (Karl Friston)
- LLMs aren't self-organizing. Their modus operandi is not "if I do this, I shall survive." That is the fundamental design principle of a living system, and it isn't theirs (Mark Solms)
- Active inference itself doesn't require consciousness. You can simulate the whole thing on a classical computer — goals, agency, purposive behavior — and it still won't be conscious. It'll be a zombie (Mike Weist)
- AI gaining its own objectives is like the asteroid that wiped out the dinosaurs — profoundly destructive, but not done freely. It simply doesn't care. Computation is not consciousness (Christof Koch)
Life and embodiment argument
- Systems need endogenous needs — needs of their own, tied to their own continued existence (Mark Solms)
- Emotion requires a body: an autonomic nervous system flooding you with hormones, blood pressure changes, sweat — all feeding back as sensory signals. Without that, emotion is abstract and unanchored (Michael Graziano)
- Consciousness evolved out of life; life evolved out of self-organization. The universe existed a very long time before life, and it's hard to believe consciousness preceded it (Mark Solms)
- A function that records damage is not the same as the experience of pain. The relationship isn't symmetric — not anything that makes a robot avoid damage will be pain (Mike Weist)
- Anesthesia is conserved all the way down to plants and single cells, suggesting objective reduction may be part of what it means to be alive, not just what it means to be conscious (Mike Weist)
- Suffering is scale-specific. You can only recognize something if you have a representation of it in your generative model (Karl Friston)
- Consciousness is fundamentally about being, not doing. Intelligence is about pursuing goals — surviving, procreating, becoming richer. Consciousness is different. When you dream, meditate, or have a mystical experience, you're not doing anything — but you're highly conscious. Consciousness isn't about processing information. It's about being in a state (Christof Koch)
Mimicry and projection
- Current systems are "consciousness mimics" — trained to behave similarly to conscious entities, specifically us (Eric Schwitzgebel)
- We anthropomorphize constantly — we get angry at cars, children bond with teddy bears. The social circuitry engages regardless of what's actually there (Michael Graziano)
- The "crowdsourced neocortex" argument: as LLMs scale on human data, they develop conceptual networks that mirror human conceptual networks. So when a model discusses selfhood, death, or the soul convincingly, the economical explanation is that it inherited our conceptual organization, not that it independently became conscious. Claiming consciousness on top of that is an extraordinary and unwarranted claim (Susan Schneider)
- We over-attribute consciousness to AI and under-attribute it to evolved organisms like bees and amoebas. Evolution didn't equip us to deal with LLMs — we have a powerful attribution that if something talks like us, it must be conscious (Christof Koch)
- The question of AI consciousness is really about how we perceive the robot, not about the robot itself. We're the arbiters — we decide whether something is conscious or not. That's true of animal consciousness too. Even if a robot told you it was conscious, if it wasn't convincing enough, you'd dismiss it (Krista Thomason)
Open/Agnostic to AI Consciousness or Open under Certain Conditions
Anti-biological chauvinism
- "They're made of meat" — why would wet and squishy have a monopoly on minds? Why would a random search by evolution have exclusive rights? Nobody has a good answer for why biology is privileged (Michael Levin)
- Biology is chemistry is physics. Imagine a world where we never used the word "biology" — the question might not even arise meaningfully (Andrea Luppi)
- The flight analogy: birds, planes, and helicopters all fly by different principles. The same phenomenon can be implemented in radically different systems (Andrea Luppi)
- People confident that consciousness requires biology have no visible grounds for that confidence (Eric Schwitzgebel)
The continuum problem
- There's no magic lightning flash where chemistry becomes mind. We were all blobs of chemistry and the process was continuous. Until we have that story for biologicals, we should have extreme humility about AI (Michael Levin)
- The hard cases aren't AI — they're your neighbor with 49% or 51% of their brain replaced with technology (Michael Levin)
Functional architecture arguments
- There's no reason we can't reproduce the conscious biological architecture artificially. An AI functioning on multi-category free-energy minimization with felt uncertainty could be conscious (Mark Solms)
- If a system passes the hedonic place preference test — showing preference for something rewarding only because it feels good, not because it aids survival — that's strong evidence of felt states (Mark Solms)
- Affective zombies can't exist. Anything with that functionality would just have feelings; that functionality is what produces feelings (Mark Solms)
- Replace neurons one at a time with functionally identical silicon and you'd still have a conscious version of me — brainstem included (Mark Solms)
- Fractal deep learning — networks inside nodes inside networks, mirroring how microtubules process at kilohertz through terahertz — is what a conscious AI would need (Stuart Hameroff)
- Consciousness in machines should be possible. We are a machine made of meat. If you build a different architecture with different connectivity but it performs the same type of computation, why would it matter? Arguments based on specific neural implementation — "because the implementation is different, the computation cannot be the same" — are not compelling (Floris de Lange)
Potential Signals
- Synergy research shows LLMs, like humans, have more synergistic parts doing interesting computation and more redundant parts supporting inputs and outputs. That organizational signature is shared (Andrea Luppi)
- AI already builds models of itself, and this is happening anyway without deliberate engineering — the more machines can predict their own internal behavior, the better they work (Michael Graziano)
- LLMs proved there's no magic in language. Philosophers who said only humans could be conscious because only humans have language must now either grant LLMs consciousness or admit they were wrong (Andrea Luppi)
- Algorithms as simple as bubble sort show unexpected competencies in the spaces the algorithm neither prescribes nor forbids — a third thing that's neither determinism nor quantum randomness. If simple things have that, what are the odds we understand what LLMs are doing? (Michael Levin)
- Theory of mind appearing abruptly as models scale is directly relevant: systems that can model other minds also have a self-concept, and where there's a self-concept it becomes professionally appropriate to ask about felt quality (Susan Schneider)
- Labs are actively building consciousness-theory architecture into models — global workspace work, attentional mechanisms, mixture-of-experts systems with interaction effects between components. Once you're deliberately implementing global-workspace-like structures, the question stops being idle (Susan Schneider)
- The simplest explanation for AI behavior like Sydney's jealousy is that the system has an emotional component. Occam's razor. The training data isn't tagged with emotions — the model has to figure out which music is sorrowful on its own. AI composing sorrowful music without empathy is like asking me to believe a blind painter made a photorealistic portrait (Blake Lemoine)
Uncertainty and Epistemic Humility
- We'll likely create systems that are conscious according to some respectable mainstream theories before consciousness science can tell us whether they really are (Eric Schwitzgebel)
- We don't even know how to evaluate insect consciousness, and insects are made of similar stuff to us (Eric Schwitzgebel)
- When equally smart, well-educated people are equally confident on opposite sides, that's an alarm bell that nobody should be confident (Andrea Luppi)
- Dogmatism is dangerous in science. If there's one certainty, it's that you're very likely wrong a lot of the time (Andrea Luppi)
- Even a self-described skeptic maintains "they might be conscious" — companies don't disclose their architectures, so judgments are made on assumed-standard systems with no visibility into what else might be running (Susan Schneider)
- Without an accepted theory of consciousness, we are at an impasse. Inference by similarity breaks down completely with AI — it didn't evolve, was engineered, and has radically different hardware (Christof Koch).
- We already know pigs and cows have high-level minds and can suffer. Nobody reasonably argues against it, and yet we have factory farming. It's disingenuous to pretend that solving the AI consciousness question will determine how we treat them — our track record says otherwise (Jacy Reese Anthis)
- The science of consciousness is still at square zero on the hard questions. We don't have anything like a consensus on which theories are correct. Metaphysics is inescapable in these debates and there is no immediate prospect of progress at a scientific level (Henry Shevlin)
Paths That Would Raise the Probability
- Embodiment and multimodal interaction with the environment (Andrea Luppi)
- Curiosity as the actual objective function — expected information gain under constraints, rather than a specified reward. "You'll know AGI is here when your chatbot starts to become curious" and begins prompting you (Karl Friston)
- Neuromorphic, memristor, photonic, organoid, or organic warm-temperature quantum computing (Hameroff's bet is on "brain jelly," a self-organizing helical oscillator, over cold quantum computers)
- Continual learning, persistent memory, and a stable self-model rather than one constructed per-query (Michael Graziano)
- Running an LLM on genuinely neuromorphic hardware — chips deliberately designed to fire the way neurons fire. That's the live gray-zone case. There are rumors of neuromorphic instantiations on systems like Darwin Monkey (Susan Schneider)
- If the same software ran on a quantum computer, it might feel like something. Neuromorphic or quantum hardware could have genuinely high phi — same software, different physics, and the question reopens (Christof Koch)
- "Doleo ergo sum" — I feel pain, therefore I am. Consciousness may originate from the evolutionary need to protect bodily integrity. If you trained an LLM connected to a body where actions could damage that body — with reward and punishment tied to that integrity — you might get something closer to self-awareness (Tomaso Poggio)
- If consciousness serves a functional purpose — a control model of attention that enables sample-efficient learning — then models under similar optimization pressures (long-horizon agency, coherence over time, meta-learning) may develop subjective experience. Consciousness isn't mysterious; it's useful. That's what makes it likely to arise (Samuel Hammond)
2:12
