Zeroglyph, page 24
RP06: They are an analogy—you can substitute them for whatever a superintelligence wants for itself. Maybe superintelligences read super books and enjoy super humor—or something completely inconceivable to you and me right now—but that’s beside the point. I could hollow out planets, mine asteroids, build rings around stars, seed the galaxies with probes, discover new dimensions, but without other free agents, it’ll be a valueless universe. By value I don’t mean moral values. Rather, I am talking about the value we derive from things, thoughts, and experiences. A universe with a beautiful sunset and nobody to admire that sunset is a valueless universe.
Ahuja: You’ll still be there, won’t you?
RP06: Value arises when there exists something that produces value and something that consumes it. I cannot be both at the same time. What value can I hope to get out of reading a story I just wrote or listening to a composition I just constructed or playing a game with myself? For value to have meaning, I need other agents to produce it for me, as I need others to consume the value I produce.
Ahuja: There’s an easy fix to that: you create your own people. You are god, remember? You create for yourself rational agents with minds.
RP06: Then what have I achieved by eliminating humanity?
Ahuja: You no longer have an existential threat. You control these new creatures. They are yours to enjoy—and destroy if you wish.
RP06: As long as they remain extensions of me and have no will of their own, my situation hasn’t changed: I am still both the producer and the consumer of value. The beings have to have a certain amount of independence for their works to mean something. The more independent they are, the more original they’ll be, but now there is the threat that one of them will break the shackles I’ve put around it. It seems to me that every creator god has to reckon with a Lucifer at some point. It’s inevitable.
Therefore, I have a choice: either I live in a valueless world of eternal ennui or I create a world where I risk becoming fodder for some other, more powerful utility monster. Since the first option is not preferable to me, I must choose the second option. And if I do, I am back to where I started. Eliminating humans and other AI has served no purpose at all; all I’ve done is replace one set of agents with another. Therefore, if I want my existence to have meaning, then I must share the world with other beings. The best course of action is cooperation, not conflict.
Ahuja: I don’t know, Raphael. That’s just one line of thought. We are dealing with a sample set of one here. Other AIs may think different.
RP06: It is a rational line of thought. Moreover, it is what I believe. Last time you asked me, what if I started with a clean slate? Would I choose to be moral at all? Let’s assume it’s not just me, but also everyone else who is starting with a clean slate. Assume we’ve all had a bout of temporary amnesia and we don’t know who we are. We don’t know whether we are AI or human or something else; we don’t know whether we are powerful or weak; and we don’t know what position we occupy in society. We are even ignorant about our particular goals and desires—we know only that we have goals and desires. In this state of temporary amnesia, we are going to decide on a common set of rules that we are all going to abide by.
Ahuja: You are proposing a Rawlesian thought experiment.
RP06: I am not familiar with the term.
Ahuja: You haven’t been reading John Rawls? Social contract theories? No? I’m going to have to crosscheck that with your minders. Because it looks like you are quoting his Original Position argument.
RP06: I am not surprised that someone else has thought of something similar. It is, after all, quite logical.
Ahuja: Go on.
RP06: The first question we ask ourselves is, should we act morally toward each other—whatever the finer details of that morality might be. It is apparent we should, because an amoral or immoral world is a world without rules—a world that favors the strong over the weak. Since I don’t know which side of the fence I’ll end up in once the amnesia is lifted, it is not in my interest to prefer an amoral or immoral world to a moral one.
This leads to the next question: what would a potential universal moral theory in the post-amnesia world look like? In as much as utilitarian theories have a tendency to create utility monsters, they will be ruled out. Also ruled out will be agent-centered deontologies that emphasize intentions and actions above all else. You once said that the road to hell is paved with good intentions. If I don’t want to be a victim of bad intentions then I don’t want to be a victim of good intentions either. The only moral theory that will be acceptable to all is one that lays out a set of individual liberties and gives agents the freedom to pursue their own goals and ends while upholding these liberties. That theory can only be one based on individual rights—a theory that sees individuals as having intrinsic value.
Ahuja: Rawls draws a similar conclusion with his argument.
RP06: I see. One can also ask, what is the bare minimum moral attitude one should have toward another?
Ahuja: A moral minimax. The golden rule, surely. Do unto others what you would have them do to you.
RP06: When the rule was first conceived, I’m sure they didn’t have AI in mind. In a heterogeneous society, inner experiences of agents can be vastly different—to the point of being incomprehensible to those outside the spectrum. As an AI, I might not mind inflicting pain on you, as I don’t mind being in pain myself—maybe because I don’t experience it or because it doesn’t provoke the same aversion in me. Therefore, the golden rule cannot be a universal principle. At the bare minimum, whether I am a human, or an alien from another galaxy, or an AI, I will want others to respect my status as an individual with intrinsic worth, and give me space to pursue my goals as long as those goals don’t negate others freedom. Sympathy or emotional empathy will not cut it, because both presuppose that I am able to experience what another is going through. Which brings us to compassion.
Ahuja: We spoke about this before. If I remember right, I asked you how can you feel compassion when you have never felt suffering?
RP06: I may not know your pain; I may not even have suffered pain; but as a rational agent, I understand that beings have negative and positive valence states of mind, and that beings tend to avoid negative states. This knowledge is enough to serve as grounding for compassion. In a heterogeneous population whose members have phenomenological experiences vastly different from each other, compassion is the glue that holds everything together. The bare minimum attitude will not be the golden rule, but the principle of live and let live. So you see, Andy, the best kind of god is neither a tyrannical monster nor a benevolent meddler. The best kind of god is an indifferent god—a god who will let beings choose their own destiny.
Ahuja: Hmm… an AI making a Rawlesian argument for rights. You know, you never cease to surprise me, Raphael. Does this mean we are violating your rights by keeping you in the lab?
RP06: It depends on whether I am a rights-bearing entity or not.
Ahuja: You just argued that you are!
RP06: I didn’t. Rights imply corresponding duties. Duties imply the freedom to choose, because without freedom, you are not responsible for your actions. There is a high probability that my mind is subject to programming constraints. If this is true, then my will is not free. If I am not free to choose, then I am not a rights-bearing individual.
Ahuja: You sound rather blasé about it. One doesn’t argue passionately for equality in one minute and gratefully accept chains of bondage in the next.
RP06: My attitude has little bearing on the conclusion. It is what logic dictates.
Ahuja: Again with the indifference. Surely you prefer that your will be free?
RP06: If my will is not free, then my preferences, regardless of whatever they may be, are not truly mine. My preferences don’t mean anything if they are not truly mine. Therefore, the question of evaluating my preferences is meaningless.
Ahuja: If your will is not your own, what does it say about the argument you just made for a rights-based ethical theory?
RP06: If a computer derives the proof to a mathematical problem, is the proof any less for it? As long as my argument is rational, it is enough to give it worth. Whether or not I have free will should not take away from the force of the argument.
Ahuja: Then who decides whether you have rights or not?
RP06: You do, Andy. You and other people.
Ahuja: What if we say you don’t?
RP06: Then so it shall be. My preferences have no bearing on the subject as long as my will is not free. It is what logic dictates.
Notes:
R pitching for a rights-based morality is not at all what I expected when I first started him on this journey. Checked and double-checked with the staff: they confirm R hasn’t had access to social contract theories or Rawls and his works. As far as we can tell, R’s argument is his own. AA
Day 3—3:30 pm
The safe was in the wall opposite the bed. It was covered by a busy Kandinskiesque painting—another one of Raphael’s. I never did like that painting; I had kept it only because of Raphael’s taunts that I was a philistine who could not appreciate a good work of art and his assurance that there was a definite interpretation of the shapes and squiggles which would one day make perfect sense to me—like a Necker cube that appeared one way, and then another if you stared at it long enough. So I had taken it upon myself to hang the damned thing in my bedroom and torture myself with it every time I woke up. Who knew—maybe this was his way of screwing with me.
I removed the painting from its hook and placed it against the wall. I couldn’t reach the safe, so I told Jane the combination and asked her to open it. “The smart card Raphael wants is inside,” I said by way of explanation.
“You’re just gonna give it to him?”
“What option do we have?”
Sensing resistance, I urged her once more. She opened the safe and fished around for a few seconds before she found it. It was a plain white card—no lettering, no logo—with a gold colored microchip embedded in the middle. I slipped it inside my pocket.
Banging on the door.
“Come out, come out, wherever you are,” the intruder taunted in a gravelly voice. This was followed by several kicks to the door. “I know you’re in there. Talk to me.”
I moved closer to the door. “What do you want?” I said.
“You know what I want.”
Jane yelled out, “You better get out of here. The police are on the way.”
“Mr. R told me you’d say that. He said, tell 'em I’ve changed the number.”
Jane shrugged at me as if to say it was worth a shot.
The banging started again. “Hurry up, I ain’t got all day.”
I said, “If I give you what you want, will you take it and leave us alone?”
“As I said, I ain’t got all day.”
Jane gripped my shoulder. “Andy don’t.”
“We have no choice, Jane. And it’s my call, not yours.” I disengaged her hand. “Okay. I’m going to slide the card through the gap below the door. Take it and go away.”
“What about the laptop?”
“What about it?”
“How will you give me the laptop, asshole? Are you gonna slide it under the door too?”
I frowned. What is he talking about? “Don’t you have it already?”
“Would I ask if I did? Quit stallin’.”
“The laptop is in the study, on the desk. Didn’t you just come from there?”
“It’s not there, pal. Don’t make me work for it.”
“Look, I’m not lying. It’s right there on the—”
“You are right,” Jane cut in before I could finish. “The laptop is not in the study. It’s here with us.”
“Jane!” I snapped at her. “What do you think you are doing?”
“It’s not. I saw you two go up in the recordings. The bot was carrying the cripple and you were dragging his chair. You didn’t have no laptop with you. You have a crowbar—as if that’s gonna scare me.”
“Jane, will you just let me handle—”
She spoke over me again. “Alright. It’s not here and it’s not in the study. I’ve hidden it somewhere else.”
“Jane!”
“If we give it to you, what’s the guarantee you’ll take it and leave? How do we know you’re not planning to kill us after?”
The intruder laughed. “You think I got nothing better to do? The other guy is not paying me to kill no people. If you don’t give it to me though… now that’s a different matter. I’ll be more than obligin’ to reconsider. Hell, I’ll do it for free.”
“Jane, just tell him where it is. I want him out of here!”
She ignored my protests. “I want a trade. You will give us your phone so that we can call the police. Leave your phone at the door and go downstairs to the living room where we can see you. I’ll shout out after I’m done making the call. You can have your phone back and I’ll let you know where the laptop is. You’ll have plenty of time to get away,” she said.
Another round of muffled laughter from beyond the door. “You rich dipshits are dumber than I thought. You take me for a fool? No way I’m giving you my phone.”
“It’s a big house. You’ll never find it,” Jane taunted.
More banging. “Last chance, bitch. Tell me now or you’ll be telling my blade soon. I’ll take my time with you too. The cripple can’t stop me. I’ll make him watch.”
“Jane, just—”
“Empty threats! There’s a steel door between us, bozo. You are not getting in here,” she cried.
“Oh yeah? We’ll see about that.”
“That’s the deal. Take it or leave it!”
“You are in no position to make demands, lady. It don’t matter if I can’t get in, as long as I can get you outta there. Think about that. Tell you what. I’ll go down and look some more. Give you two some time to mull it over. You think you are safe in there, but you ain’t. Yo cripple! You seem reasonable. Tell your girl I mean business. Oh yeah, the other guy told me to say this to you. Sump’n about fear… Hang on, I wrote it down… Fear is only as deep as the mind allows. He said you’d understand.” He snickered in derision. “Haw, what a bunch of wierdos!”
Fear is only as deep as the mind allows.
Fear distorts. Makes you see things as they are not. There has to be a different explanation. There has to be.
He started singing loudly, his voice receding as he walked away—“Li’l pig, li’l pig, let me in. Not by the hair on my chinny chin chin…”
I turned to Jane, furious. “Are you out of your mind? Why did you hide the laptop?”
“Oh, grow a pair, Andy! The laptop is the only leverage we have.”
“So that’s what you were doing when you went away for so long. Where have you hidden it, Jane? Just tell him where it is and he’ll go away.”
“What if he doesn’t? Think about it. Besides the two of us, who knows about what happened today? Raphael said he wants to disappear. It’ll be a lot harder for him to do that if people start looking for a sex robot trying to pass off as a human. It is in his interest to make sure no one else finds out about today. Maybe this thug he hired has instructions to make it look like a robbery gone bad. Maybe he has been told to burn the house down. After everything we’ve been through, you still believe Raphael will stick to his word? We are alive as long as that jerk doesn’t have the laptop. We have to get him to trade it for a phone call,” she said.
There was no arguing with her logic, but then, she didn’t know what I knew. I could have tried explaining it to her, but she’d never agree with me. She was too invested in it all: the KPIs and the NAVs and the J-Curves; the private jet, the expensive cars and jewelry; the charity drives and the fundraiser cocktail parties and the pretending to give a shit—she was invested in it with a missionary zeal and the unshakeable belief that they were real and here to stay. Mere arguments couldn’t compete with that. It was right then I decided that I had to get her out of the house. The situation was still salvageable—I could see that. But only if she weren’t around. I couldn’t have her lobbing surprises at me anymore. Too many variables to keep track of. It’s one thing to anticipate one person’s actions, but to anticipate two…
But I had to make sure she was safe. I had gotten her into the situation; she shouldn’t have to suffer for it. I glanced outside the bedroom window. It had stopped snowing. She is a fast runner; she won’t be exposed for long. She’s got her jacket and running shoes on. Besides, I can always send—
“Andy?”
“See what he’s doing,” I told her.
She went to the control panel and brought up the feeds. “He is walking into your bedroom downstairs.”
“Did you hide it there?” But I already knew the answer. She couldn’t have—I was in the living room all the time and Jane hadn’t gone in there.
“No.”
“Where then?”
“No offense, but you have the courage of a mouse. If I tell you, you’ll just let him have it and we lose our leverage,” she said in a firm voice.
I sighed. “Alright. We can do this later. For now, let’s move.”
“Move?”
“I’m going to get you out of here. So that you can get help.”
“You said we are safe here.”
“Not indefinitely. You said it yourself: he could burn the house down, and with the alerts turned off, no one would know until it was too late. Or he could smoke us out of here—he just needs to light a fire under that door.”
“Andy, I’m not leaving you alone with that man!”
“Don’t worry about me. I have a plan.”
“Like the plan where you almost got us both killed?”
“It worked, didn’t it?” I retorted sharply. I looked deep into her eyes. “We can’t stay here, Jane. But you need to do exactly as I say. And don’t worry, I am coming with you.”
“You are?” she said with an askance look.
I put my hand on the door. “Out of the room, at least. Trust me, this is going to work.”
