What the Terminator Series Gets Wrong About Artificial Intelligence and Self-Awareness
Skynet from the Terminator series is the archetypal representation of artificial intelligence in science fiction. And according to the Terminator mythology, the defining characteristic of an AI that matches or surpasses humans in intelligence is the ever-elusive "self-awareness." But David Deutsch, the physicist who is often called the "father of quantum mechanics," argued in Aeon that this common conception of AI is erroneous, and the attribute of "self-awareness" is relatively arbitary.
He wrote, "Remember the significance attributed to Skynet's becoming 'self-aware'? That's just another philosophical misconception, sufficient in itself to block any viable approach to AGI. The fact is that present-day software developers could straightforwardly program a computer to have 'self-awareness' in the behavioural sense - for example, to pass the 'mirror test' of being able to use a mirror to infer facts about itself - if they wanted to. As far as I am aware, no one has done so, presumably because it is a fairly useless ability as well as a trivial one."
According to Deutsch, the defining characteristic of AI is not the ability to recognize itself in a mirror, but the ability to formulate explanations of the world in general. Or, in other words, it's not self-awareness that matters, but awareness. "AGIs will indeed be capable of self-awareness - but that is because they will be General: they will be capable of awareness of every kind of deep and subtle thing, including their own selves."
He also cleared up another misconception about the quest for a human-like artificial intelligence: that it could arise out of a traditional computer network that generates output based solely on input. Since the goal is, ostensibly, to create a computer that behaves like a brain, it "cannot possibly be defined purely behaviourally," but rather, must be able to generate thoughts, feelings, and ideas independently of stimuli. He uses the common thought experiment of a brain in a vat; theoretically, if a living brain were sitting in a vat, isolated from any external stimuli, it would still have thoughts about itself and its existence, for example, "How did I get into this vat?" Until a computer program can mimic this ability, it cannot be considered to be as intelligent as a human.