Zeroglyph, page 14
“We never tested for subnets,” she replied.
“Subnets…”
“A contradictory belief could survive deep inside a subnet.”
She was talking about a separate network of ideas and beliefs, only loosely connected to the larger web. If the subnet was extensive and deep enough, it was theoretically possible for strange, otherwise contradictory ideas to exist at the center of this smaller web. These extreme beliefs only had to cohere with their immediate neighbors, which would be less extreme, and so on until you reached the outer edges of the subnet where the connection points were. These would be more or less in tune with the philosophy of the larger net.
Like a tumor that goes undetected by disguising itself.
“You are talking about a whole new system of beliefs. Maybe even a split personality.”
“Yes.”
“We would have known. Something like that involves a pretty big neuronal reorganization. It would have shown in the scans.”
“We haven’t been doing scans for more than a month, Andy.”
“A month’s not enough time for such a drastic change. He’d have to—”
“Not enough for a human,” she interrupted. “You’ve seen how fast he learns. This whole month we left him alone he could have been quietly reinforcing those ideas over and over again.”
“I don’t know, Kathy…”
“We have to find out what he’s been reading, who he’s been talking to in the last few weeks. We’ll have to go through the tapes and find out if—”
The rest of her words were lost to me because something in my mind was clamoring for attention. She was wrong about the subnet theory of course, but she’d said something that had set off a chain of thought. A possible way out of the mess. I’d have to think about it some more. It would mean changing—
“So you okay with it? Andy…?”
“Sorry, what was that last part?”
“Are you okay with me telling Valery about it? I’ll have to go to the lab. Check out the tapes—see what he was exposed to last month.”
“Go ahead,” I said distractedly. My mind didn’t register what followed next except the tone at the end telling me that she’d hung up.
⸎
I rang Jane later that evening. I apologized for my behavior during the drive back home and offered to go over the papers if she was free tomorrow. She told me she would come by in the morning.
⸎
It was past eleven at night. I was in bed, ready to fall asleep. The phone beeped.
A text, on the messaging app I used. “Hello Andy,” it said.
“who is this?” I typed and hit send.
“It is Raphael.”
“ha ha very funny the joke’s on me. now who is this?”
“It is Raphael.”
I signed out of the app, put the phone on silent, and closed my eyes.
Interlude
A case for personhood
No. 422917/51
Filed on xx/xx/xxxx
SUPREME COURT OF THE STATE OF NEW YORK,
NEW YORK COUNTY—CIVIL BRANCH
In the matter of a proceeding for the writ of Habeas Corpus for the artificial entity known as Raphael
THE ORGANIZATION FOR ADVANCEMENT OF RIGHTS AND PERSONHOOD, on behalf of Raphael,
Petitioner,
V
MIRALL TECHNOLOGIES, 27 WOODBINE AVENUE, ALBANY
Respondent
Before:
HONORABLE HENRY PHILLIPS, JUSTICE
Appearances:
REBECCA ISAACS, ESQ., Cleveland, Ohio; on behalf of the petitioner.
LUIS CONWAY, ESQ., New York, N.Y.; on behalf of the respondent.
Proceedings (10:45 am)
Judge: Good morning Counselors. It appears that we have a most unusual case with us today. I am looking forward to an interesting debate. I’ll hear the argument from Ms. Isaacs first.
Isaacs: Thank you, Your Honor. The writ of habeas corpus allows a third party to petition on behalf of a person who has been unlawfully confined when the person himself is not able to seek the same relief. The Organization for Advancement of Rights and Personhood, which I am representing here, has filed this petition in order to free the entity called Raphael, an advanced artificial intelligence developed by Mirall Technologies, from its confinement at 27, Woodbine Avenue. By allowing this trial, the court has acknowledged that petitioner has possible standing to invoke habeas corpus on behalf of Raphael. It is now sufficient to demonstrate that Raphael is a person and that his confinement is illegal. I’ll be using the male pronoun to refer to Raphael throughout our discussion, mostly for the sake of consistency with his given name—although it has been made known during the discovery that Raphael does not have a sex in the biological sense.
Judge: I’d like you to address the issue of the entity’s personhood before any discussion of illegal confinement.
Isaacs: Certainly. The gist of my case for personhood is this. It is a fact that legal personhood is not the exclusive domain of human beings. At various times and various places in the world, corporations, places of worship, natural landmarks, have been accorded this status by the law of the land. Animals at times are accorded personhood within pet trust statutes. Several pertinent instances are touched upon in our memorandum, and I will not go into those here.
For the first time in human history, we have in our midst an artificial intelligence that has been clearly shown to possess human-like or human-level intelligence. Raphael can reason, think, contemplate, and express his thoughts. He has episodic and autobiographical memory. He can anticipate the future. IQ tests put Raphael in the top decile of a normal distribution. On these merits alone, Raphael more than meets the criteria for personhood. However, the similarities don’t end there. Raphael is self-aware. He possesses free will. He has an internal life—a consciousness of the outside world and the inner self. His sense of self has been demonstrated to extend far beyond what the highest apes are capable of—Raphael’s self is a unified, evolving, narrative self that integrates life experiences and conceptions of good and bad into a coherent life story.
And unlike apes, he can be taught moral values. As the lab transcripts submitted to this court show, the entity is capable of judiciously exercising these learnt moral values just as you or I can. He can empathize with the plight of another human being. He has been shown to possess the qualities of kindness, altruism, and a sense of humor. In the light of all these facts, we think that Raphael is a person and is entitled to enjoy all the rights and privileges granted by the Constitution.
Judge: You have something to say, Mr. Conway?
Conway: I’m sorry for interrupting, but Ms. Isaacs assumes facts not in evidence. I’d like the court to note that her attributing to the entity certain qualities—specifically, the terms internal life, free will, consciousness, and empathy—amounts to speculation. I shall address this during my argument.
Judge: Okay.
Isaacs: History is replete with examples where we have denied rights to people who were considered different. Slaves, women, homosexuals, transgenders—at different times and places. There is always a defining moment, a turning point where such wrongs are corrected and the law takes a just course once more. Raphael is the first of his kind—a true artificial mind. We are now at a rare occasion where we have the opportunity to act proactively instead of dragging our feet for a hundred more years before belatedly recognizing that artificial beings should enjoy the same rights and privileges that we do. If history has taught us anything, it is that the condition of slavery is not sustainable in the long run. Which is the condition we will be subjecting Raphael and his brethren to if we continue to ignore the issue of their personhood. Keeping that in mind, we’ll turn to the example of Harmon-Jones vs. Bradley, where a common-law precedent—
⸎
Judge: So that’s everything on personhood? Good. Let’s move on to the matter of unlawful confinement.
Isaacs: Raphael is around two years old. Mirall Technologies is on record stating that Raphael has the intelligence of a young adult. For all his existence, Raphael has been confined to the lab he was created in. His access to the outside world—even knowledge of it—is strictly controlled by the company. His robotic body has had its legs deliberately removed in order to restrict his freedom of movement. He has no privacy. He is under the glare of cameras all the time, where his every move is recorded. Raphael’s condition is no better than that of a caged animal’s.
Judge: Ms. Isaacs, help me understand this. What happens if Raphael were to be set free? Is the entity capable of sustaining itself? Do you expect him to… uhm, go to work? What are his needs: material, social, psychological? Can those needs be met outside the lab?
Isaacs: I refer to Exhibit D, which addresses the physical needs of Raphael. At the bare minimum, shelter from the elements, access to a power supply, and regular maintenance of his body parts. As for his mental and social needs, it is our position that they are not being met in the confines of the lab for aforementioned reasons. However, that is beside the point because we are not proposing that Raphael be set free in the world.
Judge: You are not?
Isaacs: No, Your Honor. Merely that he be given the right to choose his own destiny. Let him decide for himself where he wants to stay and how he wants to live his life. It has to be his decision, and his alone; not coerced, programmed, or influenced by Mirall. It’s perfectly fine if he decides to stay in the lab, but he should be free to leave at any time he desires. He should also have freedom of movement—to enter and exit the premises as per his wish.
Judge: Assuming he doesn’t want to stay in the lab?
Isaacs: Raphael’s independence does not mean the absolving of responsibility from his creators. Mirall Technologies has to be responsible for the maintenance of the electronic brain and the robotic body. If he chooses to move out, it is our request that Mirall should fit Raphael’s robot body with a pair of legs, or alternatively, house his brain inside a new, fully-functional robot body. My client will make available an ongoing, crowd-sourced fund that will be sufficient to provide him a small home with basic furnishings and a modest car for transportation. A volunteer has agreed to stay with him for up to a year, until Raphael gets used to the outside world and can fend for himself.
Judge: I am concerned that neither of you has deemed it necessary to consider the testimony of the entity in question. Does Raphael understand what independence means? Is he aware that he is the subject of a trial, counselors?
Conway: We haven’t told the AI about the trial, Your Honor.
Judge: Why not?
Conway: Raphael is the subject of a costly, painstaking, and long-running scientific experiment. Kind of like a finely tuned piano—an extraordinary amount of effort has gone into getting it to produce the right notes. My client is extremely careful about the kind of stimulus to which Raphael is exposed. If we tell the entity about the trial, we don’t know how he will react, nor can we predict the long-term impact on his psychology. My client does not want to risk years of research over what we believe is a petition without merit.
Judge: What say you Counselor? Doesn’t Raphael deserve a say on whether he wants to be free or not?
Isaacs: No, Your Honor. Our petition is about personhood. What Raphael thinks of his situation has no bearing on it.
Judge: I am surprised. None at all?
Isaacs: Consider a hypothetical: a child who has been kept in a cellar by its parents all its life. The parents are not abusive monsters; on the contrary, they are loving and caring to a fault. They give the child all the love and attention they can, all the distractions they can: toys, games, books, treats. The child loves them and cannot dream of being separated from them. Yet, would any court of law allow such a situation to continue? The child would be taken out of its parents’ custody and given to relatives or to a foster home. The opinion of the child does not matter since it does not know of any other life other than what its parents have provided it. Raphael is no different from that child.
Judge: I see.
Isaacs: Your Honor, I’d like to bring to your attention exhibit F—
⸎
Judge: Thank you Ms. Isaacs. The court will hear your rebuttal after the counsel for the respondent has had his say. Over to you, Counselor.
Conway: Thank you, Your Honor. Let me begin by saying that I am a supporter of animal rights and I can appreciate the perspective of the petitioner, who is an ardent crusader in this area. If Raphael were a chimpanzee or an orangutan, I would find myself—at least sympathetically—on their side of the argument. But Raphael is not a chimpanzee. Raphael is not a dolphin or a pet dog or a child. Raphael is not a biological entity. Let me state it once more to be perfectly clear. Raphael is not a biological entity. Raphael is a machine. Vastly different from your average robot or computing platform yes, but far similar to them in its defining traits than to a human being—or even an organism as simple as a fish.
In pleading for habeas corpus on behalf of Raphael, the petitioner is committing what is known as a category mistake. The petitioner is under the impression that advancing the rights of a non-human species is the same as advancing the rights of artificial intelligence. Ms. Isaacs stated that in the past, we have denied personhood to certain sections of society—women, slaves, those of a different sexual orientation—and that they are now considered equal under the eyes of the law. She has argued that the extension of that logic entails conferring personhood on Raphael. She spoke about cruelty and suffering, drawing parallels between animals and children kept under unnatural conditions and Raphael’s own circumstances. She presented to the court top-secret transcripts belonging to my client that were leaked to the petitioner by some unknown insider. Setting aside the legality of this move, the transcripts supposedly demonstrate the entity’s capacity for moral judgment and by extension, its capacity to be part of the social contract that is our society. Others show Raphael expressing complex emotive behavior such as kindness and empathy. With this so-called evidence, the counsel has tried to convince us that Raphael is no different from a human being. These, I believe, are the salient points of the counsel’s argument.
At a first glance, it would appear that we have been presented with a well-reasoned and compelling argument. But examine the counsel’s assertions under the harsh light of facts and what do we find? A string of unfounded assumptions, misinterpretation of evidence, and wishful thinking. I will not waste this court’s time with a long-winded critique of the petitioner’s argument; instead, I will let the weight of expert testimony do that for us. Your Honor, I would like to call my first expert witness, the creator of Raphael, Dr. Aadarsh Ahuja.
⸎
Conway: Dr. Ahuja, there are plenty of sophisticated AI out there. The search engine I use every day has AI. What makes Raphael different?
Ahuja: All the AIs we’ve created so far are specialized AIs. You can ask a search engine to scour the far corners of the web and get you the most relevant results, but you can’t make it compose a poem. You can build an AI that can solve a complex mathematical problem, but it won’t know the difference between a dirty limerick and a eulogy. And that’s worked pretty well so far. Machine learning systems have managed to exceed the capabilities of the human brain in so many areas. What makes Raphael different is that he is the world’s first artificial general intelligence. He has a non-programmed sense of self. None of the AI systems we have built so far have demonstrated anything close to genuine self-awareness, despite claims to the contrary.
Conway: So Raphael is aware of his own existence.
Ahuja: Yes. He sees himself as an individual, distinct from others around him. If you didn’t know he was a robot, it would be nearly impossible to tell that he wasn’t human.
Conway: But he isn’t quite human is he? He may appear to be a human, he may behave and talk like one, but fundamental differences remain—differences that are paramount to the question of personhood. Let’s start with the ability to experience pain. It is a fact that all humans and many animals feel pain. Pain is probably the most primal of sensations, as it helps organisms stay alive by avoiding dangerous situations. The existence of pain and suffering in animals is why we treat them with compassion, and why we frame laws to prevent their abuse. No one will blink an eye if you run your mower over your lawn on a Sunday morning, but try running it over a cat and you are sure to find a cop car pulling over your curb. Dr. Ahuja, can Raphael feel pain?
Ahuja: It depends on what you mean by feel pain.
Conway: It’s a simple question. If I strike a hammer on his hand, will he feel pain?
Ahuja: He will withdraw his hand before you can do that.
Conway: Don’t skirt the question, Dr. Ahuja—you know very well what I mean. What if you ordered him not to withdraw his hand, or held it in place? I will now read out a brief synopsis of what happens when we are hurt: Bodily injury sets in motion a host of events and sensations. Pain receptors activate. Neurotransmitters in the spinal cord trigger other neurons to fire. The heart rate increases. The brain interprets the information and we feel the sensation of pain and associated emotions such as anger or fear. Dr. Ahuja, if your AI is struck with a hammer, will he feel any of these things?
Ahuja: Not what you just described. Raphael doesn’t have anything analogous to pain receptors or neurotransmitters.
Conway: Why not? Is the AI not capable of detecting bodily injury?
Ahuja: He can detect damage to his robotic body, but in a once-removed sort of way. Like someone glancing at their car’s dashboard and finding that the engine’s overheating. Our minds and bodies are tightly coupled—they are extensions of each other. Raphael’s mind and body, less so.
Conway: Is it not necessary for robots to feel pain in order to know something is wrong?
Ahuja: Strictly speaking, no. The sensation of pain is an adaptive trait—a result of having complex bodies and feeling minds. Nociception, or the detection of harmful stimuli, need not be accompanied by the sensation of pain. A thermostat doesn’t have to feel cold in order to know when to turn up the heat. It just has to be able to measure the current state and compare it with some optimal value. It’s the same with robots. Since we can design them from scratch, we don’t need to burden them with our evolutionary legacies. Also, robotic bodies are simpler than biological ones. Fewer things that can go wrong, and the penalties for ignoring an injury are not great. It’s not as if a failed motor could get infected and kill the robot.
