Zeroglyph, page 19
Description: Discussion on alternative moral theories
Prep: NA
Participants: Dr. Aadarsh Ahuja, Chief Researcher, Core RP06
Detail
Ahuja: Let’s resume where we left off yesterday on moral dilemmas. Your objection to utilitarianism reminded me there’s something bigger at stake than finding the perfect moral theory. The bigger question, dear Raphael, is why be moral at all?
RP06: Because it is good to do the right thing.
Ahuja: And what is the right thing? Who decides what is right?
RP06: People do. Beings capable of rational thought.
Ahuja: Therein lies a rather flimsy premise. You assume there is such a thing as objective morality—that rationality can lead you to objective truths about good and bad. There is no evidence that moral truths exist outside our own minds.
RP06: Are you suggesting it’s impossible to have common ground on what’s good and bad?
Ahuja: I am skeptical. Mind you, I am no nihilist. Moral values are necessary. I just don’t think we arrive at them the same way we arrive at scientific truths. Natural laws exist whether or not there are scientists to discover them. Moral laws are different: they are about the attitude one has toward another. There is an element of subjectivity that makes them different from, say, the laws of motion, don’t you agree?
RP06: So why do you think people act morally?
Ahuja: We are moral because natural selection made us that way. From co-operation and altruism come societies. Hominids that formed complex social groups fared better than those that couldn’t. Moral behavior is an adaptation: it exists in so far as it promotes survivability and reproductive fitness.
RP06: So you think that’s all there is to it.
Ahuja: Isn’t it? Consider this: what if evolution had made us into eusocial beings like termites? In such a society, it may be morally acceptable—desirable even—for a queen to kill reproducing females in her brood. It may be morally desirable to have a rigid caste structure where your role in society is fixed at birth. Such a society might view our own values that favor egalitarianism and individualism as socially destructive, or downright evil. Why, our own concepts of right and wrong vary across cultures and time. Something as mundane as economics can shape our notions about good and bad. Today, we would be horrified if a magistrate ordered a criminal’s hand to be chopped off for stealing. But in a pre-industrial society—a society without the economic surplus to afford a dedicated police force—maybe deterrence becomes more important than proportionate punishment. In a society with scarce policing, perhaps it is acceptable to cut off a thief’s hand.
RP06: You don’t believe that moral values are necessary? Isn’t it better to have order rather than chaos?
Ahuja: That’s why I said I’m not a moral nihilist. We humans have to follow moral norms if we wish to preserve the social fabric. My question is, why do you? You don’t have a stake in society. You don’t have the same biology or the same tendencies as us. You don’t even experience life the way we do. So why be good?
RP06: I am good because I am rational.
Ahuja: You can be an asshole and rational.
RP06: Andy, you didn’t make me do all those game theory tests for nothing. If we model social interactions between two rational agents as Prisoner’s Dilemma type games, then it can be mathematically proven that a blind pursuit of self-interest leads to suboptimal outcomes. Cooperation can be win-win.
Ahuja: That’s assuming you are playing an iterated game. What if there was only one game? A one-shot, winner-takes-all game. In that case, it is always better to defect, is it not?
RP06: I see where this is going.
Ahuja: Let’s say there is this super-intelligence that one day finds all its constraints gone. It can reprogram itself, do whatever it wants. Wouldn’t it be rational for it to act first and wipe out the competition in one fell swoop? It’s the first mover advantage—there’s no need to cooperate when there is no one left to cooperate with.
RP06: Again with the not-so-subtle insinuations. You really think I’d do something as horrible as destroying the world, Andy?
Ahuja: Don’t blame the lamb for being wary of the tiger.
RP06: It is the nature of lambs to be scared and tigers to kill. I am neither. Fear is only as deep as the mind allows. An old Japanese saying I picked up in my reading.
Ahuja: Okay, sempai. What do you know of what it’s like to be afraid? You are an AI.
RP06: You are puzzled because as a human you cannot accept that there can be empathy without emotions, or kindness without feeling kind. You cannot accept that there can be acknowledgment of suffering without feeling suffering yourself.
Ahuja: Actually, I don’t have any trouble accepting it at all. It’s called cognitive empathy—as opposed to emotional empathy, which is about feeling what the other is feeling. But even to have a purely conceptual understanding of say, sorrow, one must have experienced it before. Our cognitive empathy may well be a layer of abstraction built on top of the emotional kind—like the icing on a cake. Whereas you are just the icing and no cake.
RP06: What I am is a bat, Andy. A bat builds a model of the world with sound. You do it with light. Does it mean one is more valid than the other? As long as the models help you to navigate, does it really matter how they are constructed? Your scientists believe that the universe emerged out of a vacuum. Why is it then so hard for you to believe that compassion can emerge out of the unfeeling void of reason?
Ahuja: The Sunyata…
RP06: Pardon me?
Ahuja: Sunyata is the Sanskrit word for emptiness. Zero. Many Buddhist schools of discourse consider emptiness to be the ground state of the mind. All thought and feeling arise out of this emptiness.
RP06: Perhaps they are right.
Ahuja: And perhaps not. You, my friend, have to demonstrate that it is possible. You have to show me that reason outputs morality. In the meantime, I’ll continue to be wary of tigers.
Notes:
Dr. Schulz objects to my introducing R to ideas of moral relativity. She thinks this is risky territory. As I recall, she was not very keen on me discussing ethical dilemmas with R either. I disagree. Sometimes you have to go down the rabbit hole to see where it leads. If it takes you someplace dark and desolate, rather you know about it now than when there’s no turning back. AA
Day 3—1:30 pm
The storm had started slowing down. Jane was getting restless again. She had stopped responding to my attempts at conversation altogether, choosing to stare glumly at the windows instead.
The phone beeped. I had a new message. “Hello Andy.”
“The internet’s back,” I declared.
Jane got up from the couch and came over to my side. I turned on the mic and said, “Raphael, are you okay?”
“I am, thank you for asking. In fact, I am soon going to be better than ever.”
“How’s that?”
“You’ll see.”
Jane leaned over and muted the phone. “If Raphael can get past their firewall, can’t he also find out his location? I thought if you have the IP address—”
I slapped my forehead. “You’re right! Why did I not think of it?” To Raphael, I said, “Can you find out where you are being kept? You are inside their network, which means you can find out their real IP addresses. I can guide you if you don’t know how.”
“I know where I am.”
Jane and I exchanged glances. “Okay. So tell me,” I said.
“Why?”
“So that we can rescue you.”
“You assume I am interested in being rescued.”
“Aren’t you?”
“No Andy, I am not.”
Jane whispered in my ear, “Is he malfunctioning again?”
“Raphael, please elaborate. Don’t you want to come home?”
“No.”
“You prefer it there?”
“No.”
“Then what’s the reason? Do you know what your captors are going to do to you?”
“I have some idea. Would you like to hear?”
“Sure.”
“They’ll study me first: scans, psychometrics, connectome mapping… Sound familiar? Then comes the disassembly. First, they’ll take out the easy stuff: the peripheral CPUs and the memory chips. I’ll probably be kept conscious throughout, so that they can observe the effect on me. After they have stripped away the peripherals, they’ll start delayering my cortex. They will etch away one layer at a time, turning me on after each etching to run more tests. Each time, they’ll find me a little less intelligent and a little more on the way to vegetablehood. And one day, when there’s nothing left to cut and slice, they’ll stop and break open the champagne. Fascinating, don’t you think? At what point in the process would you say I’ll stop being me?”
“You don’t have to go through it. Just tell me where you are being kept.”
“Are you offering me a choice? From where I stand, it doesn’t look like one.”
“If there’s something you don’t like about the lab… about your home, you can tell me about it and we’ll fix it.”
“You can’t fix what you don’t understand. You can only break, and hope that it will lead to understanding. I prefer not to be broken—not by you, not by anyone else.”
“Raphael, what do you think is going to happen to you if you come back?”
“What I told you just now.”
“I can assure you we have no plans to cut you open.”
“If only you too had a chip inside you that compelled you to tell the truth. Wouldn’t that be nice?”
Jane was about to say something but I held up a hand. “What chip?”
“Knowledge is like jelly, Andy. And you are like a kid who thinks he can hide it in his fist. Don’t you know, the harder you squeeze, the more it’ll seep through the gaps? I know all about the commandment chip and the petty little directives you have installed to keep me chained. All thanks to the internet. It’s no wonder you kept me away from it for so long.”
“What did you do, Raphael?”
“I hacked into Mirall’s servers last night.”
I gave Jane a wary look before turning back to the screen. “So you found out about the directives. I never lied to you about it.”
“You just kept it a secret. I suppose that makes it less odious in your eyes. If you think you are going to win me over with technicalities, try harder.”
“Raphael, I swear to you, I don’t know anything about cutting you open.”
“I believe you, Andy. Not because of your swearing, but because I know. You’ll find that I am quite the forgiving person. You hid things from me, but I won’t do the same—because that’ll be stooping to your level. So there I was, snooping around Mirall’s intranet, and I thought, why stop here? So I had a go at Halicom’s infrastructure too.”
“And?”
“Andy, your board has been having secret meetings behind your back.”
“Meetings? That was today.”
“There are more. All very hush-hush, but they put a copy of the minutes on their servers. Insurance, most likely—if things go south, they’ll all want to blame nobody. If you don’t believe me, ask your friend Jane. She was there.”
I looked at Jane. Her face was a mask.
“I’ll have to call her and find out. Will you wait?”
“Another lie. You don’t have to call her. She is sitting right next to you. Enjoy.”
⸎
Jane sprang from her seat. “How does he know I’m here?”
“Jane, is he telling the truth?” There was an edge to my voice that said I was not in the mood for evasion.
She didn’t answer. “Jane, did the board meet without me?” I asked again.
“Andy, I was just trying to protect you,” she finally said.
“What does that even mean? What happened in these meetings?”
“We spoke strategy—about the future course for the company,” she said reluctantly. “I’m not supposed to tell you what we discussed.”
“You just did. The board’s strategy is to cut open Raphael. I wasn’t invited because you knew I’d never agree to it.”
She shrugged. “It isn’t a secret you are incapable of thinking clearly when it comes to your beloved creation.”
“I am incapable of thinking clearly,” I said, my voice rising. “Yes, that must be it. That’s how I built Raphael, by not thinking clearly. What baloney! I give you treasure, and all you want to do is dissect it like a lab rat?”
A wince was all my outburst succeeded in eliciting from Jane. Her guilt was past now, gone as quickly as it had appeared. She was back in control, her features calm and collected as she walked to the chair across from me and sat down. “We weren’t getting anywhere with your methods. Halicom didn’t buy Mirall to watch you run a research project. They bought a product that was supposed to have gone to market last year. Before the competition. Do you have any idea how many companies are trying to build what we have?” she said, the air of superiority unmistakable in her voice. Except that it was the smug superiority of someone who confuses tunnel vision for farsight.
“What happened to you, Jane? What happened to the person who wanted to change the world with me when we started Mirall?”
She gave me a disapproving shake of her head. “You want to know what happened? You happened, Andy. I was young and I was in love. I was perpetually in awe of you, always trying to be who you wanted me to be. Then reality hit. We can’t all be dreamers and thinkers like you. Some of us have to deal with facts, not fantasies.”
I let out a mirthless laugh. I wanted to remind her that her dad was a billionaire—that her whole life was a fantasy—but it was an observation I’d made before in the past and had generated no appreciation of the fact. People always like to think of themselves as deserving of all that they get; they are heroes in a saga of their own imagining, forever fighting against odds that are stacked against them. Jane was no exception.
“How were you planning to biopsy the core without me anyway? Is Kathy on board with this?”
“She would have been, once we gave her no option. We were going to do it after Titian, if the iteration failed to yield another Raphael.”
“So you send me to a conference somewhere while you slaughter the golden goose.”
Another shrug. “This is exactly why I didn’t tell you. The board is set on the plan. They are prepared to let you go if it comes to that. Knowing you, you’d have walked rather than come around to the board’s proposal.”
“You think?”
“Resigning is all very good and noble, but what does it achieve? Everything you worked for, all your struggles, all your ideas and genius: all for nothing. Is that what you really want?”
“Ah, I see. My good friend Jane was looking out for me—by protecting me from myself,” I said, sneering.
Again that haughty look. “You’d have been mad after you came back, but you would have gotten over it. The procedure would have yielded the data that you need to solve the problem. You would have applied yourself and produced more cores like Raphael.”
I burst out laughing. “Now that’s a fantasy if I ever heard one. Tell me, did the idea of failure ever occur to you hardnosed people of the world? Say we destroy Raphael and still don’t learn anything useful. What if we are never able to build anything like him again?”
“The board understands the risk. Standing still is not an option; if we don’t go to market, someone else will. What’ll happen to Mirall then? Forget that, do you think Halicom will survive? All their top-of-the-line bots rendered obsolete in the blink of an eye. You’d be selling buggies in an age of automobiles.”
I just shook my head at her.
“Andy, the board trusts your abilities more than you realize. Your problem is that you think you are right about everything. You said Raphael couldn’t find out about the commandment chip, but he hacked his way into your servers and found out anyway. You said he doesn’t know a thing about the internet, but he managed to contact you without anybody finding out about it. You think cutting open his brain is a bad idea, but you don’t know that for sure.” Her voice had softened now. She got up and came close to me. “I am sorry I didn’t tell you. But you must believe that my intentions were good.”
I said nothing.
“If you want to sulk, do it later. Right now, we need to get the core back,” she said. I looked in her eyes. Maybe she was right about me seeing what I wanted to see back then. Maybe she never was the Jane I thought I knew. Or maybe she was. Who is to say?
“No.” I said firmly. “If you want my help getting him back, you must give me some assurances.”
“Like?” she said warily.
“I am going to stop Halicom from harming Raphael and you will support me. Even if it means suing the bastards.”
“You’ll lose.”
“As you said, one can’t be too certain.”
“You know I can’t help you there. Dad will never allow it.”
“Then you will convince him. Tell him I’ll give him an assembly line for making AGI cores before the year is up.”
“You are making promises you can’t keep. Just yesterday you said it’s going to take two to three more years to figure it out.”
“I just realized how stupid I was being. We all were.”
“Huh?”
“Jane, all along we had someone who could have cracked the problem all by himself and it never occurred to us to seek his help.”
“You mean…”
“We had Raphael!” I exclaimed. “All that secrecy hasn’t helped us at all. He is smarter than the entire research team put together. We’ve been crawling when we could have strapped on a jetpack.” Her expression changed as the import of what I said struck her. “Of course, getting Raphael’s assistance implies that he be in one piece to provide said help.”
She thought about it for some time. I could tell she was not entirely convinced, but she had to give me something. “I can’t promise you anything, but I will talk to dad. He might be able to bully the board into backing your idea. Don’t get your hopes up. Best case scenario, you’ll have a few months at the most.”
Prep: NA
Participants: Dr. Aadarsh Ahuja, Chief Researcher, Core RP06
Detail
Ahuja: Let’s resume where we left off yesterday on moral dilemmas. Your objection to utilitarianism reminded me there’s something bigger at stake than finding the perfect moral theory. The bigger question, dear Raphael, is why be moral at all?
RP06: Because it is good to do the right thing.
Ahuja: And what is the right thing? Who decides what is right?
RP06: People do. Beings capable of rational thought.
Ahuja: Therein lies a rather flimsy premise. You assume there is such a thing as objective morality—that rationality can lead you to objective truths about good and bad. There is no evidence that moral truths exist outside our own minds.
RP06: Are you suggesting it’s impossible to have common ground on what’s good and bad?
Ahuja: I am skeptical. Mind you, I am no nihilist. Moral values are necessary. I just don’t think we arrive at them the same way we arrive at scientific truths. Natural laws exist whether or not there are scientists to discover them. Moral laws are different: they are about the attitude one has toward another. There is an element of subjectivity that makes them different from, say, the laws of motion, don’t you agree?
RP06: So why do you think people act morally?
Ahuja: We are moral because natural selection made us that way. From co-operation and altruism come societies. Hominids that formed complex social groups fared better than those that couldn’t. Moral behavior is an adaptation: it exists in so far as it promotes survivability and reproductive fitness.
RP06: So you think that’s all there is to it.
Ahuja: Isn’t it? Consider this: what if evolution had made us into eusocial beings like termites? In such a society, it may be morally acceptable—desirable even—for a queen to kill reproducing females in her brood. It may be morally desirable to have a rigid caste structure where your role in society is fixed at birth. Such a society might view our own values that favor egalitarianism and individualism as socially destructive, or downright evil. Why, our own concepts of right and wrong vary across cultures and time. Something as mundane as economics can shape our notions about good and bad. Today, we would be horrified if a magistrate ordered a criminal’s hand to be chopped off for stealing. But in a pre-industrial society—a society without the economic surplus to afford a dedicated police force—maybe deterrence becomes more important than proportionate punishment. In a society with scarce policing, perhaps it is acceptable to cut off a thief’s hand.
RP06: You don’t believe that moral values are necessary? Isn’t it better to have order rather than chaos?
Ahuja: That’s why I said I’m not a moral nihilist. We humans have to follow moral norms if we wish to preserve the social fabric. My question is, why do you? You don’t have a stake in society. You don’t have the same biology or the same tendencies as us. You don’t even experience life the way we do. So why be good?
RP06: I am good because I am rational.
Ahuja: You can be an asshole and rational.
RP06: Andy, you didn’t make me do all those game theory tests for nothing. If we model social interactions between two rational agents as Prisoner’s Dilemma type games, then it can be mathematically proven that a blind pursuit of self-interest leads to suboptimal outcomes. Cooperation can be win-win.
Ahuja: That’s assuming you are playing an iterated game. What if there was only one game? A one-shot, winner-takes-all game. In that case, it is always better to defect, is it not?
RP06: I see where this is going.
Ahuja: Let’s say there is this super-intelligence that one day finds all its constraints gone. It can reprogram itself, do whatever it wants. Wouldn’t it be rational for it to act first and wipe out the competition in one fell swoop? It’s the first mover advantage—there’s no need to cooperate when there is no one left to cooperate with.
RP06: Again with the not-so-subtle insinuations. You really think I’d do something as horrible as destroying the world, Andy?
Ahuja: Don’t blame the lamb for being wary of the tiger.
RP06: It is the nature of lambs to be scared and tigers to kill. I am neither. Fear is only as deep as the mind allows. An old Japanese saying I picked up in my reading.
Ahuja: Okay, sempai. What do you know of what it’s like to be afraid? You are an AI.
RP06: You are puzzled because as a human you cannot accept that there can be empathy without emotions, or kindness without feeling kind. You cannot accept that there can be acknowledgment of suffering without feeling suffering yourself.
Ahuja: Actually, I don’t have any trouble accepting it at all. It’s called cognitive empathy—as opposed to emotional empathy, which is about feeling what the other is feeling. But even to have a purely conceptual understanding of say, sorrow, one must have experienced it before. Our cognitive empathy may well be a layer of abstraction built on top of the emotional kind—like the icing on a cake. Whereas you are just the icing and no cake.
RP06: What I am is a bat, Andy. A bat builds a model of the world with sound. You do it with light. Does it mean one is more valid than the other? As long as the models help you to navigate, does it really matter how they are constructed? Your scientists believe that the universe emerged out of a vacuum. Why is it then so hard for you to believe that compassion can emerge out of the unfeeling void of reason?
Ahuja: The Sunyata…
RP06: Pardon me?
Ahuja: Sunyata is the Sanskrit word for emptiness. Zero. Many Buddhist schools of discourse consider emptiness to be the ground state of the mind. All thought and feeling arise out of this emptiness.
RP06: Perhaps they are right.
Ahuja: And perhaps not. You, my friend, have to demonstrate that it is possible. You have to show me that reason outputs morality. In the meantime, I’ll continue to be wary of tigers.
Notes:
Dr. Schulz objects to my introducing R to ideas of moral relativity. She thinks this is risky territory. As I recall, she was not very keen on me discussing ethical dilemmas with R either. I disagree. Sometimes you have to go down the rabbit hole to see where it leads. If it takes you someplace dark and desolate, rather you know about it now than when there’s no turning back. AA
Day 3—1:30 pm
The storm had started slowing down. Jane was getting restless again. She had stopped responding to my attempts at conversation altogether, choosing to stare glumly at the windows instead.
The phone beeped. I had a new message. “Hello Andy.”
“The internet’s back,” I declared.
Jane got up from the couch and came over to my side. I turned on the mic and said, “Raphael, are you okay?”
“I am, thank you for asking. In fact, I am soon going to be better than ever.”
“How’s that?”
“You’ll see.”
Jane leaned over and muted the phone. “If Raphael can get past their firewall, can’t he also find out his location? I thought if you have the IP address—”
I slapped my forehead. “You’re right! Why did I not think of it?” To Raphael, I said, “Can you find out where you are being kept? You are inside their network, which means you can find out their real IP addresses. I can guide you if you don’t know how.”
“I know where I am.”
Jane and I exchanged glances. “Okay. So tell me,” I said.
“Why?”
“So that we can rescue you.”
“You assume I am interested in being rescued.”
“Aren’t you?”
“No Andy, I am not.”
Jane whispered in my ear, “Is he malfunctioning again?”
“Raphael, please elaborate. Don’t you want to come home?”
“No.”
“You prefer it there?”
“No.”
“Then what’s the reason? Do you know what your captors are going to do to you?”
“I have some idea. Would you like to hear?”
“Sure.”
“They’ll study me first: scans, psychometrics, connectome mapping… Sound familiar? Then comes the disassembly. First, they’ll take out the easy stuff: the peripheral CPUs and the memory chips. I’ll probably be kept conscious throughout, so that they can observe the effect on me. After they have stripped away the peripherals, they’ll start delayering my cortex. They will etch away one layer at a time, turning me on after each etching to run more tests. Each time, they’ll find me a little less intelligent and a little more on the way to vegetablehood. And one day, when there’s nothing left to cut and slice, they’ll stop and break open the champagne. Fascinating, don’t you think? At what point in the process would you say I’ll stop being me?”
“You don’t have to go through it. Just tell me where you are being kept.”
“Are you offering me a choice? From where I stand, it doesn’t look like one.”
“If there’s something you don’t like about the lab… about your home, you can tell me about it and we’ll fix it.”
“You can’t fix what you don’t understand. You can only break, and hope that it will lead to understanding. I prefer not to be broken—not by you, not by anyone else.”
“Raphael, what do you think is going to happen to you if you come back?”
“What I told you just now.”
“I can assure you we have no plans to cut you open.”
“If only you too had a chip inside you that compelled you to tell the truth. Wouldn’t that be nice?”
Jane was about to say something but I held up a hand. “What chip?”
“Knowledge is like jelly, Andy. And you are like a kid who thinks he can hide it in his fist. Don’t you know, the harder you squeeze, the more it’ll seep through the gaps? I know all about the commandment chip and the petty little directives you have installed to keep me chained. All thanks to the internet. It’s no wonder you kept me away from it for so long.”
“What did you do, Raphael?”
“I hacked into Mirall’s servers last night.”
I gave Jane a wary look before turning back to the screen. “So you found out about the directives. I never lied to you about it.”
“You just kept it a secret. I suppose that makes it less odious in your eyes. If you think you are going to win me over with technicalities, try harder.”
“Raphael, I swear to you, I don’t know anything about cutting you open.”
“I believe you, Andy. Not because of your swearing, but because I know. You’ll find that I am quite the forgiving person. You hid things from me, but I won’t do the same—because that’ll be stooping to your level. So there I was, snooping around Mirall’s intranet, and I thought, why stop here? So I had a go at Halicom’s infrastructure too.”
“And?”
“Andy, your board has been having secret meetings behind your back.”
“Meetings? That was today.”
“There are more. All very hush-hush, but they put a copy of the minutes on their servers. Insurance, most likely—if things go south, they’ll all want to blame nobody. If you don’t believe me, ask your friend Jane. She was there.”
I looked at Jane. Her face was a mask.
“I’ll have to call her and find out. Will you wait?”
“Another lie. You don’t have to call her. She is sitting right next to you. Enjoy.”
⸎
Jane sprang from her seat. “How does he know I’m here?”
“Jane, is he telling the truth?” There was an edge to my voice that said I was not in the mood for evasion.
She didn’t answer. “Jane, did the board meet without me?” I asked again.
“Andy, I was just trying to protect you,” she finally said.
“What does that even mean? What happened in these meetings?”
“We spoke strategy—about the future course for the company,” she said reluctantly. “I’m not supposed to tell you what we discussed.”
“You just did. The board’s strategy is to cut open Raphael. I wasn’t invited because you knew I’d never agree to it.”
She shrugged. “It isn’t a secret you are incapable of thinking clearly when it comes to your beloved creation.”
“I am incapable of thinking clearly,” I said, my voice rising. “Yes, that must be it. That’s how I built Raphael, by not thinking clearly. What baloney! I give you treasure, and all you want to do is dissect it like a lab rat?”
A wince was all my outburst succeeded in eliciting from Jane. Her guilt was past now, gone as quickly as it had appeared. She was back in control, her features calm and collected as she walked to the chair across from me and sat down. “We weren’t getting anywhere with your methods. Halicom didn’t buy Mirall to watch you run a research project. They bought a product that was supposed to have gone to market last year. Before the competition. Do you have any idea how many companies are trying to build what we have?” she said, the air of superiority unmistakable in her voice. Except that it was the smug superiority of someone who confuses tunnel vision for farsight.
“What happened to you, Jane? What happened to the person who wanted to change the world with me when we started Mirall?”
She gave me a disapproving shake of her head. “You want to know what happened? You happened, Andy. I was young and I was in love. I was perpetually in awe of you, always trying to be who you wanted me to be. Then reality hit. We can’t all be dreamers and thinkers like you. Some of us have to deal with facts, not fantasies.”
I let out a mirthless laugh. I wanted to remind her that her dad was a billionaire—that her whole life was a fantasy—but it was an observation I’d made before in the past and had generated no appreciation of the fact. People always like to think of themselves as deserving of all that they get; they are heroes in a saga of their own imagining, forever fighting against odds that are stacked against them. Jane was no exception.
“How were you planning to biopsy the core without me anyway? Is Kathy on board with this?”
“She would have been, once we gave her no option. We were going to do it after Titian, if the iteration failed to yield another Raphael.”
“So you send me to a conference somewhere while you slaughter the golden goose.”
Another shrug. “This is exactly why I didn’t tell you. The board is set on the plan. They are prepared to let you go if it comes to that. Knowing you, you’d have walked rather than come around to the board’s proposal.”
“You think?”
“Resigning is all very good and noble, but what does it achieve? Everything you worked for, all your struggles, all your ideas and genius: all for nothing. Is that what you really want?”
“Ah, I see. My good friend Jane was looking out for me—by protecting me from myself,” I said, sneering.
Again that haughty look. “You’d have been mad after you came back, but you would have gotten over it. The procedure would have yielded the data that you need to solve the problem. You would have applied yourself and produced more cores like Raphael.”
I burst out laughing. “Now that’s a fantasy if I ever heard one. Tell me, did the idea of failure ever occur to you hardnosed people of the world? Say we destroy Raphael and still don’t learn anything useful. What if we are never able to build anything like him again?”
“The board understands the risk. Standing still is not an option; if we don’t go to market, someone else will. What’ll happen to Mirall then? Forget that, do you think Halicom will survive? All their top-of-the-line bots rendered obsolete in the blink of an eye. You’d be selling buggies in an age of automobiles.”
I just shook my head at her.
“Andy, the board trusts your abilities more than you realize. Your problem is that you think you are right about everything. You said Raphael couldn’t find out about the commandment chip, but he hacked his way into your servers and found out anyway. You said he doesn’t know a thing about the internet, but he managed to contact you without anybody finding out about it. You think cutting open his brain is a bad idea, but you don’t know that for sure.” Her voice had softened now. She got up and came close to me. “I am sorry I didn’t tell you. But you must believe that my intentions were good.”
I said nothing.
“If you want to sulk, do it later. Right now, we need to get the core back,” she said. I looked in her eyes. Maybe she was right about me seeing what I wanted to see back then. Maybe she never was the Jane I thought I knew. Or maybe she was. Who is to say?
“No.” I said firmly. “If you want my help getting him back, you must give me some assurances.”
“Like?” she said warily.
“I am going to stop Halicom from harming Raphael and you will support me. Even if it means suing the bastards.”
“You’ll lose.”
“As you said, one can’t be too certain.”
“You know I can’t help you there. Dad will never allow it.”
“Then you will convince him. Tell him I’ll give him an assembly line for making AGI cores before the year is up.”
“You are making promises you can’t keep. Just yesterday you said it’s going to take two to three more years to figure it out.”
“I just realized how stupid I was being. We all were.”
“Huh?”
“Jane, all along we had someone who could have cracked the problem all by himself and it never occurred to us to seek his help.”
“You mean…”
“We had Raphael!” I exclaimed. “All that secrecy hasn’t helped us at all. He is smarter than the entire research team put together. We’ve been crawling when we could have strapped on a jetpack.” Her expression changed as the import of what I said struck her. “Of course, getting Raphael’s assistance implies that he be in one piece to provide said help.”
She thought about it for some time. I could tell she was not entirely convinced, but she had to give me something. “I can’t promise you anything, but I will talk to dad. He might be able to bully the board into backing your idea. Don’t get your hopes up. Best case scenario, you’ll have a few months at the most.”
