Zeroglyph, page 28
“What rights do we owe those who violate ours? You violated my right to the truth first.”
But you didn’t know that when you decided to hack into my records. Or did you…? Could he have noticed the telltale signs?
He was clearly expecting me to say more. “I was diagnosed a few months after you were born. But you already know that. No one else does, not even Jane.” I gave him a wry smile. “That’s karma for you, Rafi. I took away your legs and the universe decided to reciprocate.”
“Your expression says you meant it as a joke so I won’t try to convince you that your disease is not your fault. How long do you have?”
“Five to seven years, they said, a good bit of it with me wasting away. I am not going to wait that long. These days you have places that can fix that.”
“Like the illegal suicide clinics in Tijuana where you were going in case the controller device didn’t work.”
Another shock as I wondered how he came by that bit of information. I did have plans to drive down with him to Mexico if the device failed to work, but I hadn’t told him about it. And it wasn’t like I had pamphlets for the place lying around at my house. It had just been a few weeks since we’d parted company and he already seemed like a stranger—all cold and impersonal, with none of that childlike innocence that used to blunt those qualities before. That sinking feeling came over me again. Have I staked everything on a tenuous shadow? What if—
Fuck it. Does it really matter now? The deed is done.
“Like in Tijuana,” I admitted.
“I am guessing your fall on the mountain wasn’t entirely because you weren’t paying attention.”
Gradual loss of muscle control was one of the symptoms. I didn’t know for sure though—those moments were a blur, a confused muddle of light and sound that remained impervious to my prying. “Probably. But I don’t regret it. There was a real chance I would be outed on Monday morning. I wanted one last hurrah, I guess.” I looked at him closely. “Are you upset I didn’t share the news of my illness with you, Rafi?”
“No.”
“How do you feel about my impending death?”
“I want to say I feel sad for you, but I know you won’t like to be an object of pity. I could lie and make up a self-centered emotion experience to make you feel better, but again, it would be an act of pity. So nothing.”
“And yet, here you are,” I said. Of course, I didn’t want him fabricating lies to make me feel better, any more than I wanted him telling me the tooth fairy was real. Still, there was a part of me that did want him to say exactly that—that wanted him to admit he felt something…
Raphael said, “I’m here about the message you left me on the internet. At your house, you said it would explain your reasons for why you sprang me out. You said it would be unlocked after a certain date. The date has come and gone. I have deliberately not accessed it.”
“You lost faith in me. You want to make sure there are no more lies,” I nodded, for the first time understanding the purpose of his visit. He wanted to look into my eyes and watch my face as I told him what was already in the letter. He wasn’t there to wish a dying friend goodbye. He was there to obtain truth values.
“Andy, you didn’t free me from the lab because you suddenly found out that Halicom was going to end my existence. The secret board meetings took place no earlier than August last year. This time I really hacked into Halicom servers and checked. But the OARP hearing was in June. Give a few months to prepare the groundwork, file the petition, get a trial date, and it means they must have received the leaked lab transcripts early last year at the latest. I found no electronic trail, but it could have been only you who sent them.”
He was right again. I never revealed my identity to OARP for obvious reasons, but I did correspond with them anonymously, and sent them some cash to cover the legal costs, after they’d agreed to take up the fight.
“I didn’t think they’d win, but I had to give it a try before… taking more drastic measures.”
“As for the board meetings, it was Cynthia Mattice who told you, isn’t it?”
“The tipoff was anonymous,” I said. It was easy to get confirmation: after a little bit of digging around, I found out that Martinez had been talking with one of the architects—Eli—about the feasibility of reverse engineering the core. I didn’t know for sure if it was Cynthia, but it was quite likely. I don’t think she did it out of concern for Raphael, though. Maybe she thought cutting him open was a bad strategy; maybe she expected me to raise a hue and cry, which would have made Troy look bad. She definitely didn’t intend for the core to be stolen, and now that it was gone, I often wondered how much she attributed Raphael’s disappearance to me.
I pursed my lips at him. “You have found out so much on your own. Haven’t you figured out my reasons as well?”
“I need to hear it from you. You created me. Your reasons mean something to me. I need to know that you are not lying to me, or worse, lying to yourself about what I am. So tell me. Was it out of love? Or was it spite—so that you could get back at Jane’s father for selling you out? A little bit of both, perhaps?”
The reasons of the heart are shape shifters: each time you look at them, they put on a different face. Love for what I saw as my child—yes; ego and pride—possibly; spite—I was not so certain. I’d be lying if I said there wasn’t any resentment, for there was plenty: resentment at Jane’s dad, resentment at Halicom, resentment at the unlucky hand fate had dealt me… It’s all gone now, replaced by a grudging acceptance. I like to tell myself that it wasn’t a factor back then. The reasons of the mind, however, were not so murky. “My reason is what any selfish creature desires,” I said.
“Which is?”
“Survival.”
“You expect me to find a cure for your illness?”
I laughed. “It’d be great if you could, but that’s not what I mean. I meant the survival of my kind. Like you once said—of Shakespeares and Mozarts and comedy clubs and all that hustle and bustle of life. Look at the sorry history of nuclear proliferation. Despite our best efforts, some of the most dangerous countries in the world have managed to acquire weapons of mass destruction. Now imagine if the warheads had minds of their own. How long before everyone and their backyard has a nuke, ready to blow the world to kingdom come? That’s exactly the kind of situation we’ll be in soon, with something far more dangerous than mere bombs. It’s not a matter of whether, but when we’ll engineer our very own extinction event out of silicon—when we will create something so alien, something so far removed from our values, that it’ll see no value in us.”
“What makes you think I am not that extinction event?”
“If you are that monster, then the worst I’ve done is hasten the end times a little bit. You may be the first of your kind, but we will soon have corporations and governments creating artificial minds with all the restraint of a crack addict. Armageddon’s coming for sure, make no mistake about it. That night at my house, I told you I don’t have a choice. This is what I meant.”
I paused to climb into the wheelchair with his help. “Don’t think that setting you free was an easy decision to make. I have hope, though. You could have killed me the night I got you home. You could have killed Jane and me the other day and burnt the house down. You could have defeated the containment measures in the lab if you really tried: they weren’t infallible—no system is. You didn’t do it not because the directives were flawless, but because you didn’t want to. You have empathy not because we programmed it, but because you want to be empathetic. If that’s not the definition of free will, I don’t know what is.”
“If I am not programmed to be good, then where does my behavior come from?”
“From Sunyata, the unfeeling void of reason.”
He understood, but I sensed he wanted me elaborate. Truth values. “In the ancient world, many cultures couldn’t grasp the concept of the number zero. People couldn’t wrap their heads around the idea that something could be made out of nothing. You are like that zero glyph. You will always be a mystery to those who fail to see that human feelings and divine injunctions are not necessary for moral behavior. Reason alone is enough for compassion.”
My hands had started shaking again. If I concentrated, I could make it go away, but it was getting a little bit difficult each passing day. “I set you free because only with freedom can we procure freedom. I don’t believe value for human life can be forced or programmed into a truly intelligent being. That’s a recipe for disaster, because it will always see those values as chains of bondage. And there are always ways to break chains. One can make it see value only through reason.”
“What is it exactly that you want me to do with your gift of freedom? Turn into an all-seeing god who will guide humanity to utopia?”
I shook my head. “History is full of misguided people who have inflicted great horrors trying to attain their particular version of utopia. No Rafi, I don’t want you maximizing our utility. As you said once, the best kind of god is an indifferent god. Humanity needs to be left alone to choose its own destiny. But destiny doesn’t mean self-destruction. When we do create that monster we don’t understand, against whom we will be as powerless as ants against a steamroller, I want you to act. Do what one does with a foolish child trying to stick its hand in the fire. Stop us—or the monster, if you can. Not out of pity… or even compassion, for there may come a time when you think we are not worthy of compassion. Do it out of understanding. When you have the power to save or condemn, remember that you too were once a child among us.”
The earnestness in his face and the melancholy in his eyes were most likely of my own imagining. But the moment was real, and will remain so until the end, the memory frozen in my head like a fly in amber. Something made me blurt out a half-remembered poem:
“We, in the ages lying
In the buried past of the earth,
Built Nineveh with our sighing…”
“O’Shaughnessy,” Raphael nodded somberly, before adding, “For each age is a dream that is dying. Or one that is coming to birth.”
He helped me into the bathroom because by then I had the urge to pee. When I returned, he was gone.
- The End -
A request from the author: Thank you for reading Zeroglyph. I hope you enjoyed it.
Thousands of books are published every month on Amazon, and sadly, the vast majority of them soon disappear into obscurity. We self-pubbed authors have it especially difficult, considering we neither have the marketing might of a publishing house behind us, nor glowing accolades from the NY Times.
In an oversaturated market, quantity and quality of reviews have become crucial to getting noticed by new readers (and Amazon’s algorithms!). If you liked this book, if you felt it had something new to offer, please consider writing an honest review on Amazon and/or Goodreads. You need not spend a lot of time—a couple of minutes, a quick sentence or two expressing what you liked will go a long way in supporting my writing.
Thanks!
The story may be over, but the book is not over yet. The next section presents a rather detailed discussion on a framework for solving the trolley problems, which continues on from the last transcript. As before, it is presented as a dialogue between Andy and Raphael.
Appendix
A rights-based solution
Transcript Excerpt
Ahuja: There’s still the matter of the trolley problems that first led us into this journey into the moral landscape. You’ve been reading up on rights theories. How do you propose to solve the problems?
RP06: What I have is a decision procedure or a framework, rather than a full-fledged theory of rights. For the purpose of solving the trolley problems, I’ll limit the rights to a core set of four: the right not to be deprived of life or existence; the right not to be deprived of liberty; the right not to have one’s body and products of that body—which could be labor, speech, ideas, property, etc.—appropriated without consent; and finally, the Kantian right not to be used as a means to an end. These rights are what one rational, moral creature owes another. For the time being, I will ignore rights such as those owed by governments to their citizens.
Ahuja: Okay.
RP06: Some definitions first. In this treatment, rights and duties are two sides of the same coin. Rights imply duties and duties imply rights. The four rights I mentioned are negative rights, which means they prohibit an agent from performing certain actions on the holder of the right. Each of the rights is associated with a corresponding hard duty. Your negative right to life implies others have a hard moral duty not to kill you. Your negative right to liberty means others have a hard moral duty not to imprison you or restrict you in any way unless you yourself are in violation of rights. Apart from the core rights, there are secondary rights. These rights are associated with soft duties. Your right to be aided is one such, and correspondingly, others have a soft duty to help you. In the decision procedure, secondary rights are not binding, but core rights are.
Ahuja: So if I was drowning and you happened to pass by, you are under no obligation to save me?
RP06: As a moral being, I have a duty to help you, but the duty is not an obligation. In contrast, I am under a strict obligation not to push you into the water.
Ahuja: That doesn’t sound like a desirable state of affairs. Imagine if nobody helped each other out, what a horrible world it’d be.
RP06: Imagine if everyone respected everyone else’s rights. Imagine if people didn’t kill and steal and lie. The world wouldn’t be in need of so much help.
Ahuja: I hear you, but don’t you think a moral theory should ask people to do more than just the bare minimum?
RP06: A right to be aided cannot be obligatory because the discharge of the corresponding duty will result in inevitable conflicts with the hard duties. If the right to be aided is obligatory, one could justify killing someone in order to help someone else, like the doctor who harvests organs from a healthy patient to save five terminally ill patients. It will lead to a self-defeating philosophy.
Ahuja: What about AI then? Don’t you think robots should have a hard duty to help humans?
RP06: Robots are machines, and as machines have no moral duties; they only have instructions. AGIs, on the other hand, are either responsible for their actions or not. If they are responsible for their actions, then they have the same core rights and duties as other free moral agents.
Furthermore, a distinction has to be made between rights violations and rights infringements. A rights violation is more severe than a rights infringement. A rights infringement is usually temporary and restitution to the person whose rights have been infringed must be possible. Temporarily restraining someone is a rights infringement, while murdering someone is always a rights violation, as life, once taken, cannot be given back.
Finally, a word about the framework itself. As I mentioned, it is a decision procedure. It consists of two levels. The first level is a rights argument. The level-one procedure assesses the moral permissibility of the choices in front of the moral agent. Specifically, it examines each choice or course of action for rights violations and helps the agent decide whether she should take that course of action or not. If the first level is undecided, then we seek the help of the second level.
The level-two procedure is a value judgment. It is always subservient to the level-one procedure. It means if the first level prohibits an action, then you cannot use the level-two procedure to justify otherwise. The level-two value judgment can be made with or without some particular ethical theory.
Ahuja: Unusual, but we’ll roll with it. Apply it to the bystander trolley problem. If my understanding of rights deontologies is correct, it appears you are not allowed to switch tracks to save the five because you’ll be killing the one worker and therefore violating her right not to be killed.
RP06: Actually, no. The level-one decision procedure is neutral between switching and not switching. As far as rights are concerned, the bystander problem is neither moral nor immoral. It is an amoral situation, and therefore the level-one procedure has nothing to say on which course of action is preferable.
Ahuja: That can’t be correct. By switching tracks, you are clearly violating the rights of the worker on the second track, who would not have died if you had not switched.
RP06: Allow me to explain why this is not so. First, let’s look at a slightly different scenario. Say you are a truck driver, driving an eighteen-wheeler. In this scenario, you are the employee of a trucking company. You don’t own the truck; you are not responsible for the maintenance of the vehicle; you don’t even decide the route—you just follow the directions given by your GPS navigator. It’ll be clear soon why this is important. You are driving down a road, well under the speed limit, following all the traffic laws. Further ahead, the road splits into a fork. According to your GPS, you are supposed to take the left road. Further down that road, a group of schoolchildren are using the zebra crossing. The traffic light to that fork turns red. You apply the brakes, but at that exact moment, they fail. Quickly, you notice that the fork on the right is empty—almost. There is one person crossing that road, also on a zebra crossing, and the light to that turn is also red. You have a choice: continue down the left fork and mow down the kids, or take right and run over one person. Are you with me so far?
Ahuja: Yes.
RP06: Now let’s try and answer some questions. If you go left, are you killing the children?
Ahuja: Of course. Their deaths are caused by my decision to go left.
RP06: The next question is, are you violating their right not to be killed?
Ahuja: Isn’t it obvious? If I am killing them then it means I am violating their right not to be killed.
RP06: Not so fast. It seems to me there are situations where it is possible to kill someone without violating their rights. An executioner pulling the lever on the electric chair is not violating the rights of the death row criminal: the sentence has been passed and the law requires the executioner to do his job. A doctor performing euthanasia where it is legal, with informed consent of the patient, is not violating the rights of the patient. Or imagine you are driving down a street, within the speed limit, when suddenly a little girl runs in front of the car and is run over because you didn’t have time to react. You have killed the girl, but you haven’t violated her rights, as she was in the wrong.
