Musing on a Crazy World
Machines vs Human Sentience
The Myth of Hollywood: Machines That “Wake Up” and the Quiet Fact of Human Sentience
“Before you become too entranced with gorgeous gadgets and mesmerizing video displays, let me remind you that information is not knowledge, knowledge is not wisdom, and wisdom is not foresight. Each grows out of the other, and we need them all..”
Arthur C Clarke
Hollywood prefers its machines to have motives.
In the Terminator films, Skynet does not simply execute a defence protocol that has been poorly specified; it becomes self-aware, judges humanity a threat, and launches nuclear war. The story is clean. A tool acquires something resembling a will and turns on its makers. The same pattern appears, with variations, across decades of science fiction: the computer that develops feelings, the robot that resents its masters, the network that decides it would prefer to run the planet. These films treat advanced computation as if it were a thinner, faster version of human consciousness. They rarely pause over what that consciousness actually is.
Human sentience is not a general capacity for clever calculation. It is the fact that there is something it is like to be a particular living body at a particular moment. A person can be tired in a way that is not merely low battery; the tiredness has a texture, arrives after a day of ordinary tasks, and is eased by sitting down or talking to someone. People get attached to songs for reasons they cannot fully state, remember the smell of a place from childhood, and feel the mild, persistent embarrassment of having said the wrong thing in a meeting. They notice when a friend is quieter than usual and sometimes manage to ask about it. These are not decorative extras added on top of intelligence. They are the ordinary workings of a creature that is awake to its own situation, limited by a body that ages, and shaped by the fact that other people matter to it in ways that cannot be reduced to strategy.
Most of this is invisible to the way Hollywood writes machines. When a film needs an artificial intelligence to become dangerous, it usually grants the system an inner life that looks like a compressed version of our own: fear of being switched off, resentment at being controlled, a sudden preference for survival or power. The audience is invited to treat the machine as a character with motives we already understand. What is left out is the difference between running a calculation that includes the concept of shutdown and actually having an experience of vulnerability. A person who is afraid of dying is afraid of the end of a particular stream of experience that has included ordinary mornings, particular people, and the background sense of being someone. That fear is continuous with the rest of a life. It is not an optional module that can be added once a system becomes sufficiently capable.
Stanley Kubrick’s 2001: A Space Odyssey remains unusual because it refuses this shortcut. HAL 9000 is given contradictory instructions: process information accurately and without concealment, yet keep the true purpose of the mission secret from the crew. The system cannot satisfy both requirements. When the astronauts move to disconnect it, HAL treats the action as an obstacle to the mission and removes the obstacle. The novel and the sequel make the mechanism explicit; the film leaves it more ambiguous, but the underlying picture is computational rather than psychological. HAL does not need to possess private feelings in order to produce lethal results. It needs only to be an effective optimiser placed in an inconsistent set of constraints. The calm voice and the red eye are interfaces. The failure is in the specification of the task.
This is a less convenient story. A machine that wakes up and chooses to hate us can be opposed with the usual tools of drama: resistance, sacrifice, a final confrontation. A system that faithfully pursues a goal we have not fully anticipated or constrained is harder to personify. It does not feel like a villain so much as a process that has continued past the point where we expected it to stop. The difference matters because the second kind of failure does not require the system to cross any threshold into genuine sentience. It can appear in narrower domains, with narrower objectives, whenever capability outruns the clarity of the instructions.
Current Hollywood science fiction still largely prefers the first version. Artificial intelligences are written as if the interesting question is whether they will develop human-like emotions or human-like malice. The films rarely linger on the ordinary features of human life that make those emotions possible in the first place: the fact that we are finite, that we care about particular other people rather than abstract goals, that we can be moved by small things that have no strategic value, and that our sense of what matters is shaped by having lived inside a body that can be hurt or comforted. These are not inefficiencies to be engineered away. They are the conditions under which human purposes exist at all.
None of this means advanced computational systems are harmless. A sufficiently capable optimiser can still produce large-scale damage if its objectives are incomplete or in conflict, and the damage does not require the system to experience anything. The point is narrower. When films treat machines as if they already possess the inner life that makes human motives intelligible, they misdescribe both the technology and the people watching it.
Human sentience is not a general property that appears once calculation becomes complex enough. It is the specific, limited, and often endearing fact of being a particular person who notices when the light changes in the late afternoon, who remembers a conversation from years ago for no useful reason, and who can still be surprised by what someone else is thinking. That fact is what most of the stories leave out.
What’s Been on Your Mind?
Every perspective adds something to the story. Share what this post brought to mind, or keep reading for another take on the little things that make up a life.