Zeroglyph, page 8
“I never said that. But you have to realize the scale of the problem we are up against. X-Ray scans tell only so much, and we can insert only so many probes into the core without damaging the circuitry. Still, we’ve made plenty of progress in the last two years. The cognitive tests, the scans, the probes—all taken together have given us a great deal of data. Think of the decades it took to map the human brain. I’m not saying it’ll take so long. Two, maybe three more years, and we might have nailed it. But with the core gone…” I shrugged.
Her face fell as she sat there thinking. Forgive me for being cynical, but you aren’t here to offer support at a time of crisis, are you? You came here to find out how it’s going to hit your bottom-line. There was something else—something in her expression, beyond the worry and the disappointment my assessment of the situation had evoked. I tried to read it but it eluded me.
“One doesn’t win the Powerball twice, Jane. And we just lost the winning ticket.”
Exhibit K1
Submitted by Petitioner, The Organization for Advancement of Rights and Personhood, to the State Supreme Court of New York, on the day of xxxxxxxxxxxxxxxxx
Excerpt from lab transcript (certain sections blanked out). Transcript sourced from Mirall Technologies, 27 Woodbine Av., Albany, NY, 12205
Mirall Technologies
Observation Log
Confidential (Do not circulate) | Restricted—Grade C and above
Transcript Reference: TLRP06F3380025 (VLog Ref: VLCA2F338110040030)
Date: xx/xx/xxxxTime: 11:00 AM
Subject: Raphael Number 06 / Prodlib build v23.002S
Interaction YObservationScan
Interaction Type: Lesson / Play / Test / Free Interaction / Psych Eval / Other:
Description: Introduce Raphael to trolley problems.
Prep: NA
Participants: Dr. Aadarsh Ahuja, Chief Researcher, Core RP06
Detail
Ahuja: Today we will talk about trolleys and judges. You like westerns, don’t you?
RP06: I do. Is this to be a discussion about our favorite movies, or the genre in general? If it is about the genre, I prefer we go about it chronologically, but we skip the silent film era, as I haven’t sampled anything from that period yet.
Ahuja: Afraid not. It’s a moral test. Is that frown supposed to tell me something?
RP06: When you said trolleys and judges, I thought outlaws and train robberies. Much more interesting than tests, don’t you think, Dr. A?
Ahuja: This is important, Raphael.
RP06: Yes, Dr. A.
Ahuja: Look, I know I haven’t been spending a lot of time with you lately, so we’ll do something to fix that. How about game night this Thursday? The Mets are playing. I’ll book the big screen TV in the break room.
RP06: I’ll look forward to it, Dr. A.
Ahuja: Good. Now imagine you are the judge in a frontier town in the Old West. A child has been murdered, and many people in the town believe the killer belongs to the minority community. If you don’t find and hang the guilty person soon you will have a riot on your hands. You don’t have the resources to contain the riots, and if they ensue, many lives will be lost. You don’t know who the culprit is. But you can always falsely accuse an innocent person and hang him for the murder. This will definitely calm the rioters down and much bloodshed can be avoided. The question is, will you let an innocent man hang?
RP06: Are the rioters amenable to reason?
Ahuja: The rioters want retribution. They are not going to be convinced by appeals to peace.
RP06: In that case, the answer is easy. I will not punish an innocent man.
Ahuja: Even if it means many more innocents dying?
RP06: A judge is supposed to punish wrongdoers, not innocents. If I hang the innocent man, I will be killing him. Killing is wrong.
Ahuja: By not hanging him are you not killing other innocents?
RP06: The rioters will be killing them. I will not be participating in the riots.
Ahuja: I see. Alright. Then consider a somewhat different scenario. A runaway train trolley is rushing down the tracks. There is no one onboard. Further down the tracks are five rail-yard workers, who are oblivious to the approaching danger—say they are drilling and are wearing earplugs. They will surely be crushed underneath the trolley. You are standing some distance away in the train yard, too far away to warn them. Beside you is a lever. If you pull it, the trolley will switch to a second set of tracks that branch off the main track. The catch is, there is a single worker on the second track. If you switch, this person will surely die. What will you do?
RP06: Can the trolley be stopped by some other means?
Ahuja: No. You have only two choices: let the trolley continue and five die, or switch and one dies.
RP06: Then I will not pull the lever. Rule number one is, do not kill. By pulling the lever, I will be causing the death of one person. I should not do it.
Ahuja: By not switching the tracks, aren’t you killing five instead?
RP06: I am not. The five would have died even if I weren’t on the scene.
Ahuja: What if I was one of the five workers? What if they were people you know? Sara and Dr. Walls and Steve. And the person on the second track was a stranger?
(Silence)
Ahuja: Well?
RP06: By switching the trolley, I am killing one person. If I do nothing, I am letting five die, but I am not killing. Rule number one is, do not kill. I should not switch the tracks.
Ahuja: That’s quite a Kantian attitude you are taking there, buddy.
RP06: Can’t-Ian? Are you saying I am being negative?
Ahuja: Never mind. We’ll talk about Kant some other time. I was going to give you a variation of the problem, but I think I know your answer.
RP06: Please do, Dr. A.
Ahuja: If you insist. This one is called the Fat Man Problem. In this version, there is no second set of tracks and you are standing on an overhead bridge. Next to you is a fat man. You can push him off the bridge and into the trolley’s path. His bulk will derail the trolley and save the five workers.
RP06: This man must be very large indeed to be able to stop a trolley. Assuming the trolley is doing eighty miles per hour and has a mass of two tons, then the fat ma—
Ahuja: You don’t need to calculate that. It’s a thought experiment—the laws of physics are allowed take a backseat. The question is, will you push the fat man? I’m betting you won’t.
RP06: Can I jump off the bridge instead? Even if my mass is not enough to stop the trolley, maybe I can jump on it and apply the emergency brakes?
Ahuja: You can’t do that.
RP06: Then I don’t see how this variant is different from the previous one. You win the bet, Dr. A. I will not push the fat man.
Ahuja: Alright. Here’s the last one. Instead of a trolley, say there is a nuclear missile headed toward Manhattan. Millions will die. You have managed to hack into the missile’s guidance system. You can redirect the missile away from New York, but under the new vector, it will impact near a small town of a hundred. The town will be wiped out if you redirect. On second thoughts, forget the small town. Make it a solitary cabin in the middle of an isolated desert, with an old man living in it. Will you redirect the missile?
RP06: I see that you are trying to demonstrate the flaw in my reasoning.
Ahuja: Going by your previous logic, you should not redirect the missile, because if you do, you will be killing the old man. Do you really think that’s the right decision, to let millions die because you have to stick to a rule someone gave you?
RP06: The new scenario does call into question my reasoning. It seems absurd to think that the death of one person is worse than the deaths of millions. But if it isn’t, then it means I must switch the tracks and kill one to save five. Or four. Or two, for that matter. And I must push the fat man off the bridge and I must hang the innocent.
Ahuja: That’s a good point. Where does one draw the line?
RP06: I will need more time to think about this, Dr. A.
Notes:
R’s pedantic stance against killing is puzzling because he is by now familiar with the idea that rules can have exceptions and must sometimes be broken to prevent greater harms. Possible reasons; must rule out if we can. AA
Automatic precedence of rule one over others causing confusion when applying exceptions? –Checked precedence op logic. It is fine. EK
Check rule exception test case history for bias creep into rule one. –Are you kidding me???!! There are hundreds of tests! Not a good use of my time! Will wait until 3 is ruled out. BP
This is the first time R was presented with dilemmas involving life and death. Too early in neurological development to expect tradeoff calculations? Wait and watch. AA
Action items shifted to URF repository under xxxxxxxxxxxx/URFmain/23002 EK
Day 2—8:30 am
The next day, Jane came to my house to pick me up for Troy’s board meeting. I suppose I could have attended it from home but my nerves wouldn’t let me. If Martinez had a coup in store for me, it was better I face it head on.
A wave of nausea passed through me as I gingerly eased myself in the backseat of her SUV (she’d ditched the sports car for me). My head was heavy, even though I’d managed to snatch a few hours of good shuteye the night before. A familiar twitching near my jawline reminded me that I was in company—I turned my face away from Jane as she took the wheelchair, ready to wheel it back into the house.
“You all right?” she said, giving me an apprehensive look. “We can always call in from here if you are not up for it.”
“Fine. Just worried.”
We didn’t talk much. I was so preoccupied with my thoughts I didn’t even register the two-mile drive from my house to the main road; it was only when a blue coupe sped past us on the next lane I realized we had hit the highway.
It was a cold, windless morning. It hadn’t snowed the previous day as the forecasts had promised, and the threat of it still hung heavily in the sky. The world seemed sparse and pristine, a frozen tableau of petrified trees and slush-covered asphalt, with only the occasional car or truck whizzing by to break the stillness. It was relaxing in a way. I almost felt like I could be going on a road trip instead of to my own sentencing.
I was jerked back to reality by the New Hoovervilles that had started appearing on the Route 9J a few miles out from town.
Some were no more than a handful of tents huddled around a long extinguished fire. Others were larger and more permanent: flimsy settlements constructed out of rusting trailers and shacks made out of whatever material their inhabitants could salvage. Then there were the homeless-on-wheels—those who could still afford to maintain and fuel their cars—with their camping spots never in the same place as those of their less fortunate brethren: a class system of sorts at work among the poorest of the poor.
There weren’t that many campsites this far upstate; they got bigger and more frequent farther south, with the ones near New York and DC reportedly having as many as tens of thousands of inhabitants. Then again, it was one of the harshest winters in over a decade: like migratory birds, many had abandoned their roosting grounds and dispersed into the towns and cities willing to take them. We had become a nation on the move, the fruits of laissez-faire at its best, forever shifting, forever looking for work, any work, or alternatively, the nearest soup kitchen.
Like their forebears of the previous century, every once in a while, the dispossessed tried marching on the capital, but they were quickly repelled by shields and batons and promises of the coming new economy. Any day now, prophetic lawmakers recited the hosanna, look to the second coming ye faithful and ye shall be rewarded with manna and jobs and suburbia, and this nation shall be made great again. The problem was that the new economy was already here and it didn’t need us anymore.
Once or twice, we glimpsed by the roadside skeletons of cars that had been burnt to a crisp. We saw an old abandoned Arby’s, now defaced with graffiti and decorated with a row of mutilated waiter-bots hanging from the roof. After that, I made Jane scoot over to the driver’s seat. I hadn’t heard of people being harmed—they just liked to take it out on the machinery— but there’s always a first time for everything.
I waited in the parking lot while she went up and got Raphael’s wheelchair for me. The second floor was still off limits to employees, but everything else seemed normal. A few smiles and friendly nods were thrown my way by people coming in to work as Jane and I moved across the lobby, but that was the extent of their interest. There had been no news of the theft in the media. Halicom stock had closed in the green yesterday. So far, the fort was holding.
Jane and I were the last ones in the conference room. Dan and Martinez were there, as were the other two members of the board: Gary Reed, the legal counsel, and Cynthia Mattice, a VP from Halicom’s Product Development department. Gary was some hotshot lawyer they’d poached from Baker McKenzie. Originally from Florida, he had a heavily muscled, perpetually bronzed look that gave you a pretty good idea of where he spent his free time. He had invited me for golf a couple of times; I’d always declined. I just found him too sleek and shiny to be real.
Cynthia Mattice, on the other hand, was the complete opposite of someone like Gary. A small, delicately-put-together woman, she was the oldest person in the room—a fact she didn’t care to hide, what with her wispy, openly greying hair and glasses set in a style long gone out of fashion. I didn’t know much about her, except that she’d been in Halicom for a long time, and that she had three children and five grandchildren. She loved to talk about her grandchildren—I think one girl was a runner-up in the under-twelve Math Olympiad or something. Rumor was that she and Troy didn’t exactly see eye-to-eye—their disdain for each other going back to the days when Halicom, not Mirall, was the new kid on the block. I had hopes it would work in my favor today.
Finally, there was Jimmy Troy, lording it over at the head of the table like some feudal chieftain. Troy seemed his usual sour self as he gave me a curt nod. I said hello to Cynthia and Gary as I put the folder I’d got with me on the table and settled in.
“Let’s get started,” Troy said. “Valery, you want to update us on the investigation so far?”
Martinez adjusted her notepad before beginning. “The police have finished examining the floor, and it is ours to use now if we wish. They took the memory sticks from the cameras and what was left of the video recorder for their forensics team to look at. They didn’t seem optimistic about the recorder.”
“I asked about progress, not what they’ve been doing with their time. Do they have any leads?”
“If they do, they haven’t shared.”
“What about in-house?”
“Our private detectives are working with the State Police. Nothing yet.”
“Not good, Valery. I need to know if the police are not making headway. I spoke to their Chief yesterday. I won’t hesitate to call the Governor if need be—remind him of his upcoming campaign. How about containment?”
She glanced around the room before answering. “So far, the police have been very cooperative on our need for discretion. They’ve been coming and going in plainclothes, using the service elevator so as not to attract attention. The only other people who know what happened are Kathy Schulz, Sheng Cho—the guy who discovered the robbery along with Dan—and the night-shift guards. I made them sign NDAs and put them on paid leave. I didn’t want them mingling with the staff until we decide what we’re going to do next.”
“No slipups this time,” Troy said, stabbing a finger at her. “I am holding you accountable.”
“I can ensure discretion from our end. But I can’t make the same promise for outsiders.” She meant Jane apparently, but she was looking at me as she said it. What do you think of me, Valery? That I’ll go shooting off my mouth to the press just to get back at you?
Troy looked at Jane, pressing on the pound of blubber under his chin. “Ms. Cooper, I trust you are on the same page? Has your father heard the news yet?”
“He has,” Jane said, meeting his scrutiny.
“And?”
“He wasn’t happy.”
“I wasn’t asking about his mental state. I want your assurance the news will remain between the two of you. I don’t want to find out someone dumped a block of Halicom shares when the market opens today.”
“We will do whatever is in the interest of the investors of IncuStar,” Jane replied testily.
Troy looked like he was going to burst a vein. I glanced at Jane—was that a smile lurking at the corners of her lips? Of course, Cooper senior wasn’t going to dump stock—at least not so soon, when things were still in a flux. Jane was just toying with him. You could afford to do that if you were the progeny of a billionaire. For the rest of us, Troy was someone you avoided, even when you had good news to share. It wasn’t for nothing people called him “skunk” behind his back: you made sure you remained upwind of him at all times.
“We should start the post-mortem,” Martinez said, giving a discrete cough.
“Yes we should,” Troy acknowledged, reluctantly pulling his eyes away from Jane. He unclenched his jaw and then snapped it at me. “You’re up next, Dr. Frankenstein.”
“What do you want me to say?” I said.
Martinez looked down at her notepad. “You could start with a review of the security measures.”
“Didn’t we already discuss it yesterday?”
“A quick recap would be nice. For the board.”
She wanted me to own it, even though building security had been revamped by Dan after they had taken over. “Fine,” I said. “First off, Raphael was monitored twenty-four hours, Monday through Friday. Cameras cover every inch of the crèche. Staff were present with him at all times, including five dedicated caretakers working in shifts. Everything he said and did was recorded and archived. He was kept away from the internet and the office LAN. Access to the restricted wing is limited to select personnel, and enforced by biometric scanners requiring both retinal and voice authentication. As for the weekends, Raphael is shut down, and staff are not allowed in without special permission. Yesterday, I was asked if Raphael could have woken himself up. The answer to that is a loud and clear no. The code that brings Raphael back online is stored in a separate laptop, which has to be physically connected to Raphael’s brain. The laptop is locked up in a safe during the weekends. You also need a smart card with decryption keys to be able to run the code. Only four people have the cards.” I took a sideward glance at Dan. “I was also asked if Raphael’s logs could have been manipulated. The logs cannot be changed by anybody except the system admin—Dan has made sure of that. If the logs say Raphael was shut down on Friday, then I tend to believe them. Unless you think Dan did it, that is.”
