Zeroglyph, page 7
“What don’t you believe?”
“That Raphael did all those things. Turning himself on as if by magic. Destroying his room. Aiding a theft. All of it. It’s not him.”
“We all saw what happened, Andy.”
“It’s not possible for him to do those things,” I said flatly.
She appraised my statement with the wary expression of a prison shrink who’s heard it all. “Andy, I can understand how shocked you must be right now, but you must snap out of it. The board will look to you for answers tomorrow. Denial is not the ticket out of this mess.”
I fidgeted in my chair. “I need to be in the lab, not here. Got to find out if there’s something they are not telling me…” Jane stood up and walked over. Kneeling on leg, she took my hand in hers. “Listen to me. This is no time to play detective. The police are on the case, and Valery is more than capable of coordinating w—”
“Screw Valery! This is my company! My creation! Who the hell is she? A goddamn pencil pusher! She’s out there helping? She’s busy digging a hole for me, that’s what she’s doing.”
“Let her,” she said softly, as if talking to a petulant child. “Don’t waste your time. Even if you cannot figure out what went wrong with Raphael, you have your hands full with the new iteration. It’s now more important than ever for you to get the tech working. As long as the board thinks you can deliver, they are not gonna fire you.”
Her hands felt clammy. I withdrew mine from hers. “You don’t understand. With Raphael gone, we are dead in the water. There’s no more Mirall and no more CEO to run it. It’s over.”
Creeping worry lines betrayed the calm demeanor she had been affecting until now. “You are exaggerating, right? What about Titian?”
“What about it? What about Rembrandt and Salvador? You remember those? Your father made us start work on the Rembrandt iteration less than a month after we found out RP06 was self-aware—before the enormity of what we’d created could properly sink in. The cores didn’t deliver—no general intelligence. Then we did Salvador. No general intelligence. After we were sold, Halicom made us do two more iterations. Zilch. Nada. You really think Titian will be different?”
The truth is, Raphael’s discovery had come as both a blessing and a curse for Mirall. Overnight, we stopped being a startup with a relatively grounded idea and became a moonshot company. We had managed to put god in a bottle and we were supposed to do it all over again.
She went and flopped herself on the couch. Her lips pressed into a corner as she put on that skeptical face again. “The old I-can’t-recreate-Raphael-until-I-know-what’s-inside-him argument. Remind me again, why do you have such a hard time replicating what’s essentially a machine?”
You don’t remember because you don’t think it’s worth remembering. The problem with Jane was that she was always in a hurry, forever doling out her time in chunks of half-hours and half-stops, always trying to get somewhere that was not here. A silent sigh passed my lips as I prepared to explain to her once again why repeating our success had proven difficult. “Raphael is not some amped-up computer. A part of him is, sure—what we call the Translation Layer: conventional processors, memory, code, drivers that interface with the robot body…everything you think of when you think computer. This is the reproducible part—you make one chipset, you can make a million more exactly the same; you write a piece of code, you can copy it countless times. It’s the stuff underneath that poses the challenge.”
Her face scrunched into a frown of concentration as she tried to recall something. “The neuromorphic layer,” she said, pronouncing the words slowly, as if they belonged to a foreign language. I nodded. “Like the brain, the neuromorphic layer is a massive neural network, only artificial; instead of neurons and synapses, it’s made up of neuristors and nanotubes and molecular assemblers.”
“Still a machine, right?”
Still a machine. Yet, for the past few decades, people had been building machines whose inner workings were a mystery, even to their own creators. “You are thinking of conventional computing, where you specify the procedure or the algorithm,” I said. “If you want a program that can play chess, let’s say, then you write a write a step-by-step procedure that will search and evaluate possible moves and win conditions. You can make the program as complicated as you wish, but you still understand it because you came up with the algorithm. A chess program built with neural networks is a different beast altogether. It’s a whole new paradigm. Here, you don’t tell the program how to win. You only tell it what a win condition looks like; you perhaps tell it how to play the game; the rest, it has to figure out by itself. It does this by repeatedly pitting itself against other players and learning from its mistakes. You might have guided it during the learning process, but more often than not, you have no idea what algorithm it is using to achieve those wins. The game-winning procedure is spread out in the internal variables and the structure of the network. You have to decipher it, which gets more and more difficult the deeper and more complex the network. In effect, you have created for yourself a black box that gives you the results you want, but you don’t really know how it’s accomplishing those results.”
“Okay, maybe you don’t know how it works on the inside but you can create a second chess program, right? And a third and a fourth. Why can’t you do it with Raphael?”
“Blame it on plasticity,” I said.
“What’s that?”
“In biology, neuroplasticity is what makes brains learn and adapt to ever-changing conditions. The human brain has about eighty-five billion neurons, each neuron linked to hundreds or thousands of other neurons via axons and synapses. As we make our way through the world—learning, acquiring skills, forming memories—new connections are constantly being made while those that are not in use are being pruned away. Mirall’s cores work along similar principles. Each core has around fifty billion neuristors. And much like axons in the brain, the interconnects between the neuristors can be dynamically adjusted, rerouted, and even grown on demand. Of course, there’s a lot more to the human brain than a bunch of neurons and synapses. The cores don’t even begin to rival that kind of complexity. But compared to the neuromorphic chips of a few years ago, they are pretty out there.”
They were more than just out there. Fabrication tech had undergone a seismic change in recent years. Before, microchips were essentially two-dimensional slices onto which circuits were etched or laid out. If you wanted to add more circuitry, you had to either increase the area of the chip or make the components smaller. Or, you could go vertical—make the chip three-dimensional. 3D VLSI was not exactly a new thing, but most techniques involved stacking chips on top of each other and connecting them with vias. There was a limit to how many you could stack before factors like heat and leakage put the crimps in your design. As much as neural-network-based computing was a paradigm shift from traditional computing, the recent breakthroughs in memristor fabrication and molecular assembly were a quantum leap away from the older ways of making chips. Instead of having many neuromorphic cores connected with a crossbar grid, you could now have one highly interconnected, “organic” core that was a lot less like a computer and a lot more like the stuff between our ears. Hardware now had the malleability of software.
“So the core rewires itself,” she said. “Why does that make it hard to produce another Raphael?”
“Because Raphael is not the core that rolled out of the fabbing facility. All our cores have the same base structure; we can make them all the livelong day. Raphael is the pattern of circuitry that evolved thereafter, during training. Say we had to clone you, as you are now. We don’t know how to do it, of course, but I believe it would involve something along the lines of scanning your brain in great detail—at the cell level at minimum—mapping out all the neurons and the interconnections, and somehow replicating this structure—the connectome—on another medium. It’s the same with Raphael. To create more like him, we first have to understand his connectome. To use an analogy, a fresh core is like the mind of newborn baby whereas Raphael is the mind of an adult.”
“If the cores are the same, why haven’t they all become sentient? All babies grow up to become persons don’t they?”
“Okay, wrong analogy. Actually, a newborn baby’s brain is not like a newly minted core. A baby’s brain is far more complex and far more intelligent. It comes hardwired for language, love, intelligence, problem solving—behaviors all encoded in our DNA. A new core is just an inert piece of hardware—a tabula rasa if you will. It doesn’t know anything, can’t do anything, It acquires useful intelligence later on, during training.”
“I still don’t see the problem. You put the cores through the same training as Raphael.”
“Don’t you think we did that?”
“Look, I’m just trying to understand here. What happens during the training?”
“Right after fab, we do function upload. Like I said, a new core is a blank slate. It has to learn everything from the very basics if it is to be of use at all in a robot. Recognizing everyday objects, navigating around obstacles, making logical inferences, processing voice commands and mapping them to actions… a whole lotta stuff. We make ten to twelve cores per iteration. If we had to train them all from scratch, that’s pretty much all we would be doing. So we take a shortcut. We load them with pre-trained neural net packages. Some we developed on our own, and some we license from outside. Image classifiers, nets that do math, language translators… Quite sophisticated too, as many have had years of training. We modify these nets to fit our cores’ architecture and then we burn them on the hardware of the core. You with me so far?” She nodded, but with the slightest bit of reluctance. “At this point, all the cores are essentially the same because we upload them with the same functions. The differences start accumulating during training.”
“Why do you have to train them if you are loading them with pre-trained modules?”
“The human brain is modular, right? There are separate centers for vision, touch, language, long-term planning, and so on. These modules are also tightly integrated and interconnected with each other. The modules in the core are not. The core can see, and it can hear, but it cannot see and hear at the same time. During training, we integrate these disparate functions, coupling them to a global workspace. The global workspace is what gives the core cohesiveness; without it, all you have is a bunch of modules doing different things, competing for the robot’s resources, pulling it in different directions.”
“Consciousness. You are giving the core consciousness,” she said, nodding to herself meaningfully.
She had clearly heard the term before. “No one knows for sure if a global workspace can give rise to consciousness. We don’t know how the brain creates consciousness or the self—if the brain creates them at all—let alone build systems that can do it. Nevertheless, as a practical model for command, control, and coordination in the massively parallel architecture of the core, the idea works quite well.”
“Yeah, whatever, but it doesn’t explain why only Raphael developed sentience. You created him, Andy! You should be able to replicate the process.”
“About that. It’s hard to say exactly how much I created him and how much he created himself.”
She wrinkled her nose at me, trying to read if I was pulling her leg. But she saw that I was serious. “What do you mean he created himself?” she said.
“An integrated core is only slightly smarter than the sum of the functions we load into it. It is about as smart as a top-end robot, which is not saying much. Raphael is an exception. Something happened during his training—something we don’t fully understand—that made him what he is. He outsmarted the adversaries.”
“Adversaries?”
“Adversarial training. It’s one of the stages of training a core. You’ve seen it. You remember how we would chain a bunch of cores together and hook them up to the mainframe? When you asked, I joked that they were whispering secrets to each other.”
“Vaguely.”
No you didn’t. “We believe Raphael developed self-awareness during adversarial training.”
She gave an impatient shake of her head. “What’s adversarial training?”
Generative Adversarial Networks, or GANs, had come a long way since their first conception, but the basic principle was the same. You take two neural networks, one the net you want to train—the generator—and the other, the adversary—a pre-trained net, usually a classifier—and then pit them against each other. It was AI training against AI. It was faster—a good discriminator could train the newbie net in a fraction of the time it took to do it by other means—and you minimized human intervention, allowing staff to focus on more important tasks. Unsurprisingly, it was also opaque; instead of one black box, you now had two, both competing with each other, sometimes in strange and fascinating ways.
“Let’s take a classic scenario. Say you want your newly created net to generate images of cats. Initially, it has no idea what a cat looks like. It takes its best guess and draws something, which, as you might imagine, is nothing more than a bunch of random pixels. We give this generated image to the adversary, a discriminator network that is good at recognizing cat images. The discriminator’s job is to examine a given image and say how likely it is that the image is that of a real cat. The generator’s job is to generate images that fool the discriminator into believing they are pictures of real cats. At first, almost all images drawn by the generator will be rejected. Whenever an image is rejected, the generator tweaks its internal parameters, so that the next image it draws is slightly different. In other words, the generator learns from its mistakes. After hundreds or thousands of such iterations, the generator eventually draws an image that fools the discriminator. But it’s not over yet. Now the programmer steps in and tells the discriminator that it has made a mistake. The discriminator then adjusts its own parameters, so that next time it won’t be so easily fooled. And the contest starts all over again. It’s like an art expert and a forger working against each other: both keep getting better at their jobs as time passes. You took a net that did not have an inkling of what a cat was, and evolved it into something that produces life-like images of cats, all without having to do the backbreaking work of training the net yourself.”
She shrugged. “This is all very interesting, but what does this have to do with Raphael? Did he become self-aware by learning to draw cats?”
I laughed. “No. The goal was something loftier.”
“What was that?”
“To pass the Turing test,” I said, smiling.
The Turing test had many formulations, one of the simplest being what Turing himself had proposed: an interrogator tries to determine which of two players—both interacting with the interrogator through text messages—is a human and which is a computer. The Turing test goal we gave to our cores was both less and more than the original formulation. More, because our robots would have to emulate a bigger range of human behavior than the ability to have a text conversation. Less, because no one at Mirall really thought that we’d create something that would actually pass the Turing test, even in its limited form. Chatbots these days are smart enough to pass for a human, but only for brief periods of time, and provided the conversation is kept within tightly defined parameters and the interacting person doesn’t know—and no one’s claiming that chatbots are intelligent. The idea was to have a robot that would appear to pass the Turing test at least some of the time—it was certainly better than making a robot that could not pass it all the time.
Of course, no computer or neural network could be a true judge of the Turing test since it would have to be human-like in the first place. A human would always have to have the final say on whether a certain pattern of behavior passed the test or not. Nevertheless, we’d found that we could use pre-trained nets to weed out the most unlikely or unintelligent behaviors, saving the researchers a lot of valuable time. For example, a discriminator net could easily train a core to identify basic objects accurately. Or move across a room without tripping over a dozen times. Or to carry on a limited, rudimentary conversation.
The process was not without its flaws. Since the discriminators themselves were incapable of passing the Turing test, they had a tendency to reject truly intelligent behavior. If a human attempted to have a conversation with a discriminator, it was likely the discriminator would reject the human—unless the human was good at dumbing herself down to its level. Which meant that a discriminator would pass an AI only as long as the AI was about as smart as the discriminator and no more. But the chances of a truly intelligent AI emerging out of the process was so remote we never really gave it any thought.
“We use chained GANs, with multiple discriminators working against one or two cores. Chaining streamlines the generation of composite behaviors—like walking and talking at the same time. A discriminator could be a specialized net, or a core from the previous generation that had gone through its own training process. Incremental evolution, you see. We set up the GANs and let them brew, sometimes for days at a stretch without intervention. It was somewhere during this time, Raphael developed self-awareness. A true mind was born, but we don’t exactly understand how.”
“So even though you put the other cores through the same training, they failed while Raphael succeeded? Is there something special about his core?”
“We don’t know. We tweak parameters in different iterations—memristor count, channel density, polymerization factors, and such. But within an iteration, the cores are more or less the same. This is not to say that they are identical. The fabbing process is imprecise by nature, so there are always variations. A butterfly effect could have magnified slight differences during later stages. Or it could be that the adversarial training descended a rare gradient that we haven’t been able to identify yet.”
“What then? We are just supposed to sit back and wait for the next miracle?”
