THE BRAIN-AS-DIGITAL-COMPUTER FALLACY

In a recent talk at the Royal Institution, Nobel prize winner Geoffrey Hinton made a shocking claim — one that, just a few years ago, would have been dismissed by most people as outright foolish: modern AI systems could already possess subjective experiences and be conscious. Namely, there would be nothing special about consciousness and sentiency, and artificial neural networks running on digital computers already have, in principle, all it takes to possess these properties. As absurd as it may sound, this view is actually the natural culmination of the fashionable computer science metaphor of the brain as digital computer. Namely, the brain is reduced to a collection of neurons that either fire or do not, directly communicating by connections called synapses. In this primitive model, neurons just performs weighted sums of numbers, that is, addition and multiplication of the impulses received by other neurons, and apply a simple threshold function to decide whether to activate. Since this basic arithmetic can be implemented by silicon transistors, the brain-as-computer narrative implies that brain functions can all be replicated by software. In particular, consciousness and sentiency must be algorithms, operating systems like Android or Windows — only more cunning. You could write a Python program that is conscious, and can feel joy or pain, provided the developers support these features in the latest release. To be sure, check the documentation!

The fact that Hinton — one of the godfathers of modern AI — confidently asserted that today’s models are already capable of being conscious reveals how deeply the metaphor of the brain as a digital computer is ingrained in modern computer science. What once sounded absurd now is purported as serious science. In this essay, we examine the metaphor from multiple angles: philosophical, computational, and neuroscientific. From each perspective, the metaphor appears to be reductionist, misleading, and deeply flawed.

ARITHMETIC CANNOT GENERATE A MIND

Long before artificial intelligence, the philosopher and mathematician Gottfried Leibniz, one of the fathers of modern computer science, understood what is wrong with the brain-as-discrete-machine argument. In 1714, anticipating by two centuries the Chinese-room-argument by the philosopher John Searle, he wrote:

“We must confess that perception, and that which depends upon it, are inexplicable by mechanical causes, that is to say, by figures and motions. Supposing that there were a machine whose structure produced thought, sensation, and perception, we could conceive it enlarged to the point that we could enter into it as into a mill. That being so, we should, on examining its interior, find only parts pushing one another, and never anything by which to explain a perception. Thus it is in the simple substance, and not in the composite or in the machine, that perception must be sought.”— Monadology, §17

In other terms, if the mind were a digital computer, then it would be possible to realize its operations through a complex systems of cogs, wheels, and solid parts pushing each other. We could construct complex windmills or enormous clocks with subjective experiences, consciousness, and feelings like joy and pain. But if there are only disconnected parts subjected to mechanical interaction, how could anything experience consciousness, perception or feelings?

Centuries later, in 1980, Searle revived this argument under the name of the Chinese Room argument, but with a crucial distinction: Searle used it to prove that machines cannot “understand”. The argument shifted from ontology to epistemology. Imagine a man locked in a room, using a book containing all the rules that a large language model follows when answering questions in chinese. It would take very long calculations to answer each question, possibly months, but nevertheless this person, with the help of the rulebook, would provide the very same answers that the LLM returns. However, this person would have no understanding of Chinese.

However, Searle’s argument, as opposed to Leibniz’s original one, raises some difficulties, because the concept of understanding is not so clear. Wouldn’t the combination of the book and the man display understanding of Chinese? If combined together they can answer exactly each question, perhaps, yes. This difficulty can be removed by returning to Leibniz’s original flavor: can one say that the book and the man have a new mind state that is not in the man and obviously not in the book? In particular, suppose that the LLM becomes fearful when answering a question: would there be terror somewhere in the man or in the book? We doubt these questions can be answered in the positive.

The reason Leibniz’s argument is more solid than Searle’s is that sentiency is a very strong property, even stronger than consciousness: for example, without consciousness one could not feel pain. The “computation” corresponding to pain must be accompanied by awareness. Can you imagine someone saying: “Oh, I am feeling an excruciating pain! Well, I must confess I am not really aware of it, in reality the corresponding computation is happening in the background like the computation that make my heart beat. But I have the suspect that somewhere in my brain something is telling me that I must feel pain. So I feel so much pain!”. That does not sound very realistic, does it? Pain is painful because it is a conscious experience. A number storing somewhere a pain level of 9/10 does not produce any pain feeling whatsoever. Interpreting the number as producing a feeling is pure anthropomorphism: one intends it to represent pain, and so one thinks it will provoke pain, which is projecting a human interpretation onto inert data.

We can now craft a final version of the Leibniz-Searle argument, drawing also on Penrose’s “The Emperor’s new mind”, which famously advances the thesis that the brain cannot be a digital computer. An artificial neural network, in the final analysis, performs only additions and multiplications. Suppose a team of humans by pen and paper makes the computations performed by a sentient LLM. In particular, assume that a certain point these humans make the sums and multiplications that make the LLM feel pain. Where would be the pain? Would all humans simultaneously feel painful cramps at the hands? What if each human makes only some operations, then mails to another human the rest of the operations to be done, and so on, until all additions and multiplications are performed? Where would be consciousness located, where would be the pain? Of course, it would be nowhere, because the brain-as-digital-computer metaphor holds water, as I now more deeply explain from the computational and neuroscience perspective.

THE TURING MISUNDERSTANDING

It turns out that there is a basic misunderstanding that affects many machine learning scientists: they believe that since the brain follows physical laws that can be modeled mathematically, then a Turing machine — the theoretical foundation of digital computing — can simulate everything the brain does. But this belief mistakes mathematical models of reality for reality itself.

Turing introduced his machine to capture what a human mathematician does when manipulating symbols with paper and pencil. A Turing machine consists of a tape, a head that reads and writes symbols, and a finite set of rules that prescribe how the head operates. It is designed to formalize symbolic computation, not physical processes. Every modern digital computer is functionally equivalent to a Turing machine: it manipulates discrete symbols according to well-defined rules.

But a Turing machine can be operated by pen and paper, for the precise reason that it is supposed to simulate a mathematician. The hardware that realizes the Turing machine is intended as a mean to speed up computation, but computationally it doesn’t do anything different. Digital computers merely accelerate, by orders of magnitude, the computations performed by Turing machines. This property is called substrate independency.

But not all natural processes are symbolic. In nature, many systems perform computations without translating inputs and outputs into abstract representations. A tree, through photosynthesis, converts sunlight into chemical energy. Your stomach breaks down food into nutrients. No part of the input nor of the output is a symbol. Can a computer digest a meal? Pour water into a laptop and see what happens.

What computers could execute is a mathematical model of photosynthesis or digestion. You could describe food with symbols and protein, fat and glucose with others. Of course, however, the simulation loses the properties of the thing being simulated. There is no more energy production.

So here we arrive at the basic error: conflating mathematical models of reality with reality itself. Yes, we can model neurons with mathematical abstractions. But the physical properties of real neurons — their chemical sensitivities, electromagnetic interactions, and perhaps even quantum behaviors — may be essential to generating consciousness. As Searle put it, a computer model of an hurricane does not get wet or blow wind. In the very same spirit, the simulation of a human brain, which involves complicated physics, could very well lose consciousness and sentiency. This leads us to the third perspective.

THE BRAIN IS MORE COMPLEX

The brain-as-digital-computer metaphor assumes that what counts is only whether a neuron fires and to which other neurons it is connected. But the biology of the brain is vastly more complex. Neurons also receive general, chemical messages — the neurotransmitters and modulators — and are sensible to electromagnetic fields called brainwaves. There are neurons whose task is to produce dopamine and flood the brain with it. This dopamine affects the way other neurons behave, and this is a crucial mechanism of reinforcement learning in the human brain. There is no trace of neurotransmitters in artificial neural networks. Yet chemical substances are fundamental in sentiency. Dopamine is connected to excitement and motivation, adrenaline with pain and stress, serotonin with satisfaction and bonding. Modern AI systems possess none of these mechanisms. Recent neuroscience suggests that brainwaves — oscillating electromagnetic fields produced by neural activity — play a crucial part in organizing cognition. As neuroscientist Douglas-Field explains:

“The textbook description of how neurons communicate—by passing signals across synapses from one neuron to the next—is easy to understand and study, but brainwave activity is far more complex. Brainwave activity is so complicated that the scientists who study it are physicists and mathematicians using sophisticated methods of analysis. (…) Electrical fields radiate in three dimensions; they change dynamically by constructively and destructively interacting with electrical fields generated by other sources; they propagate in three-dimensional vectors according to the combined strength of the fields interacting inside the brain and according to differences in electrical resistance.”

Likely, this kind of chemistry and electromagnetism are the basis of sentiency and consciousness. Since they involve quantum physics, it seems very unlikely that digital computers can simulate them without losing the properties of the system being simulated.

CONCLUSION

The idea that the brain is a digital computer, and thus a Turing machine that can be operated by pen and paper can be conscious, is untenable. It is a residue of the old, mechanical worldview, where nature is a collection of discrete particles that behave deterministically like infinitesimal billiard balls that collide with each other. Although Einstein’s relativity and quantum mechanics demolished this simplistic view of the world, machine learning experts like Hinton seem tied to the past, naively convinced that arithmetic is sufficient to produce a mind.

Federico Aschieri