Zeroglyph, page 30
Ahuja: I just thought of another scenario that presents difficulties. What if you are not so innocent in the fat man problem? Let’s say you are the signalman. You neglected to notice the five workers on the tracks and gave the green signal to the trolley. Their deaths will be your fault. Should you push the fat man in this instance?
Up the ante a bit: instead of five workers, there is a stationary train full of passengers. And instead of a trolley, you have given the signal to another train, also full of passengers. This train will surely collide with the stationary train, resulting in scores of deaths on both sides. Assume the train has a collision detection system. If you push the fat man into its path, the sensors will be activated and they will stop the train before it hits the stationary train. Assume your mass is not enough to activate the sensors, so you don’t have an option of throwing yourself in the train’s path, which would be the morally right thing to do. You have two options. Let the trains crash, and violate the passengers’ right not to be killed, or push the fat man and prevent the accident, but violate his right not to be killed.
At a first glance, this dilemma would seem to fall under category two, where you choose between violations of the same right, the difference here being only in the quantity of rights violations. But that may be a wrong way of looking at it. The passengers’ death is caused by an act of negligence; also, it is unintended. Whereas the fat man’s death is murder, because it is intended as a means to an end. Isn’t murder a more serious charge than unintentional killing by negligence? It seems there is a qualitative difference between a right to life violation by way of murder, and a right to life violation by way of negligence.
RP06: This could mean one of two things: they are either violations of the same right or violations of two different rights. It must be noted that they do entail different duties: in one, I have a duty not to murder you, which involves premeditation and intention, whereas in another I have a duty not to kill you by my negligence, which does not involve premeditation and intention. If they are different rights, then the scenario falls under category four. Since murder is definitely a more egregious violation, and the procedure is not about minimizing the number of rights violations, the duty not to murder trumps the duty not to cause deaths by negligence. Therefore, one should not push the fat man off the bridge.
On the other hand, if they are violations of the same right, then the scenario belongs to category two. Since the choice is between violating the same rights, the level-one decision procedure is undecided, as both choices are immoral. You can then use a level-two utilitarian value judgment to justify pushing the fat man and preventing the train crash.
Ahuja: What’s the correct approach? Are they the same right or not?
RP06: I don’t know. As I said, it is just a framework; it assumes the existence of an ethical theory of rights that resolves questions of conflicts between different rights. If such a theory already exists, I can incorporate it into the framework. If not, I can begin working on it.
Ahuja: I don’t think a full-fledged theory that resolves all conflicts between different rights exists. But you have made a good start. Remember once I told you that it has to be your moral outlook, not that of others. Do you stand behind the decision procedure you have just outlined? Are you convinced that it is the best of all the other alternatives?
RP06: I am. The two-level framework strikes a balance between deontology and consequentialism. Firstly, individuals are not sacrificed for reasons of greater good, as individual rights always trump utility calculations. Ergo, no utility monsters. Two, unlike a rigid Kantian deontology, it avoids the kind of moral catastrophe exemplified by the redirecting nuke scenario, or its lesser version, the innocent bystander trolley problem. Under the framework, you can justify killing a few to save many—in certain circumstances. Third, it admits that individuals are not undifferentiated parts of the whole, and therefore, allows individuals to have special duties toward themselves and those they hold a special relation with—as opposed to giving equal consideration to random strangers.
- End of Transcript -
Afterword
Of course, there is no AGI (yet) and no court case over it, and the acknowledgments I made in the Introduction section are fake—a narrative technique used to lend an air of authenticity to the story.
This book started out as a novella and then morphed into something unusual. Unusual for a work of fiction because there is a fair bit of philosophy thrown into what initially promised to be a short, fast read—a home invasion story with a twist. But then I realized that Andy needed a good justification for freeing Raphael—something that wasn’t informed by his personal feelings alone. Once freed, Raphael would also be free of all programming constraints, and Andy realizes the danger all too well, despite his love for his creation. In order to provide that justification, I had to make Raphael “prove” that a purely logical being, possibly lacking real feelings and emotions, could be good without being programmed to be so. He had to prove that moral behavior is a natural consequence of a rational mind; only then Andy could bring himself to release the AI into the world.
Easier said than done. Philosophers have struggled with questions about the objectivity of morality for millennia, and I am no philosopher. But at least something resembling a solution had to be found, and Raphael had to arrive at this by himself. He had to evolve, from simple moral rules to a more nuanced understanding of ethics, and the evolution had to be illustrated with his changing attitude to the moral dilemmas presented by the trolley problems. He would start with a fixed deontology (his Seven Rules, a recasting of Bernard Gert’s Ten Rules, from his book Common Morality: Deciding What to Do), and progress to more thoroughly reasoned approaches, in tune with his mental development.
Ethics is a complex subject; my goal was to capture the bare essence of the debate around the three main classes of moral theories—deontological, utilitarian, and rights-based—and evaluate how they fare when applied to AI. And hopefully, without dumbing it down too much. The more I researched the various ethical theories, the more I was led to reject the first two—Kantian ethics for being too rigid, and utilitarianism for its propensity to birth utility monsters—in favor of rights-based morality. Raphael could use John Rawls’ Original Position argument to demonstrate that cooperation and altruism can be an end-result of selfish but rational thought processes.
As for the trolley problems, the Two-Level Decision Procedure as described in the Appendix is my own little attempt at applying algorithmic thinking to the moral dilemmas. Professional ethicists may find it amusing, but I think there is merit in the central logic of the decision procedure, which is the primacy of individual rights over utilitarian considerations.
A word about utilitarianism, which has many passionate proponents, AI researchers included. There are plenty of refinements to Jeremy Bentham’s original formulation, which for obvious reasons cannot be covered in this book in any detail, but I find that one of the main objections to utilitarianism in general—that of utility monsters—is often dismissed as being practically inconceivable. Yet, it ceases to remain a mere academic objection when you take into consideration superintelligent AI. I am convinced that it is a terrible idea to program utilitarian ethics into beings that may one day surpass us. Raphael’s arguments against utilitarianism in the transcripts perfectly mirror my own feelings about this particular class of theories. His apprehension is understandable; he realizes that all utility monsters must live in fear of bigger utility monsters that might eat them someday (consider the volume of fiction we, utility monsters to other species, have written about AI and/or alien-caused annihilation).
Having said that, I also recognize that one can’t get rid of utilitarian calculations from moral decision-making altogether. Sometimes overall utility does matter. Hence the Two-Level Procedure. Don’t eliminate utility, but have it constrained by the rights of the individual. Isn’t that how our present society is organized, by and large? Purists may frown, but it seems to me a very practical approach to solving the trolley problems.
For the uninitiated reader who wants to find out more about this fascinating subject, there are many excellent books and online resources that explore in great detail the complexities of artificial morality and ethics in general—some of which I have included in the references section. Of particular note are the Stanford Encyclopedia of Philosophy and the Internet Encyclopedia of Philosophy—both are free to use.
The court case was inspired by Hercules and Leo, two chimpanzees whose case for habeas corpus made the headlines in 2015-16. Those passionate about animal rights may consider donating to the Nonhuman Rights Project at www.nonhumanrights.org
Lastly, my endorsing for AI a rights-based ethical theory should not be construed as a defense of neoliberal or right-libertarian political ideologies. If anything, the book highlights the pitfalls of letting corporations create artificial minds sans oversight and sans foresight. We don’t let corporations create weapons of mass destruction in the name of economic freedom and property rights; the same caution should apply to creating superintelligent AI.
Bibliography
Books
Bostrom, Nick (2014) Superintelligence: Path’s, Dangers, Strategies. 1st ed., Oxford University Press.
Gert, Bernard (2004) Common Morality: Deciding What to Do. Oxford University Press.
Nozick, Robert (1974) Anarchy, State, and Utopia. 1st ed., Basic Books
Parfit, Derek (1984), Reasons and Persons. Oxford University Press
Pastine, Ivan; Pastine, Tuvana; Humberstone, Tom (2017) Introducing Game Theory. Jamison, Kiera (ed.), Icon Books
Rawls, John (1971) A Theory of Justice. Harvard University Press
Singer, Peter (2011) Practical Ethics. 3rd ed. Cambridge University Press
Skorupski, John (ed.)(2010) The Routledge Companion to Ethics. Routledge
Wallach, Wendell; Allen, Collin (2009), Moral Machines. Oxford University Press
Research Papers, Periodicals, and Others
Allen, Colin; Smit, Iva; Wallach, Wendell (2005) Artificial Morality: Top-down, Bottom-up, and hybrid approaches. Ethics and Information Technology Volume 7, Issue 3, pp 149-155, Springer
The IEEE Global Initiative for Ethical Considerations in Artificial Intelligence and Autonomous Systems (2016) Ethically Aligned Design: A Vision For Prioritizing Wellbeing With Artificial Intelligence And Autonomous Systems, Version 1. IEEE
Kriegel, Uriah (2004) Moore’s Paradox and the Structure of Conscious Belief. Erkenntnis 61: 99-121, Kluwer Academic Publishers
LaChat, Michael R. (1986) Artificial Intelligence and Ethics: An Exercise in the Moral Imagination. AI Magazine Volume 7 Number 2, AAAI Digital Library
Moran, Richard (1997) Self Knowledge: Discovery, Resolution, and Undoing. European Journal of Philosophy Vol. 5, Issue 2, pp 141-161, Wiley
Rapaport, William J. (1986) Logical Foundations for Belief Representation. Cognitive Science 10, 371-422, State University of New York, Buffalo, NY
Sloman, Aaron (1979) Epistemology and Artificial Intelligence. Expert Systems in the Microelectronic Age, Donald Michie (ed.), Edinburgh University Press
Varden, Helga (2010) Kant and Lying to the Murderer at the Door… One More Time: Kant’s Legal Philosophy and Lies to Murderers and Nazis. Journal of Social Philosophy 41 (4):403-4211, Wiley
Gips, James (1995) Towards the Ethical Robot. Android Epistemology, K.Ford, C.Glymour, P. Hayes (Eds.) MIT Press
Sandberg, A. & Bostrom, N. (2008) Whole Brain Emulation: A Roadmap. Technical report #2008-3, Future of Humanity Institute, Oxford University
Gert, Bernard (1999) Common Morality and Computing. Ethics and Information Technology 1: 57-64, 1999, Kluwer Academic Publishers
Hughes, James (2012) Compassionate AI and Selfless Robots: A Buddhist Approach. Robot Ethics: The Ethical and Social Implications of Robotics Patrick Lin, Keith Abney, George A. Bekey, MIT Press
Bringsjord, Selmer & Taylor, Joshua (2012) The Divine Command Approach to Robot Ethics Robot Ethics: The Ethical and Social Implications of Robotics Patrick Lin, Keith Abney, George A. Bekey, MIT Press
Heuer, Ulrike (2011) The Paradox of Deontology, Revisited. Oxford Studies in Normative Ethics, M. Timmons, Oxford University Press
Nichols, Shaun (2002) How Psychopaths Threaten Moral Rationalism. The Monist 85 (2): 285-303
Foot, Philippa (1978) The Problem of Abortion and the Doctrine of the Double Effect. Oxford Review 5:5-15
Thomson, Judith Jarvis (1985) The Trolley Problem. The Yale Law Journal, Vol. 94, No. 6 (May, 1985), pp. 1395-1415, The Yale Law Journal Company, Inc.
Thomson, Judith Jarvis (2008) Turning the Trolley. Philosophy & Public Affairs 36, no.4, Wiley Periodicals
Miller, David (2012) Are Human Rights Conditional? Centre for the Study of Social Justice Working Papers Series, SJ020, University of Oxford
Beavers, Anthony F. (2012) Moral Machines and the Threat of Ethical Nihilism. Robot Ethics: The Ethical and Social Implications of Robotics Patrick Lin, Keith Abney, George A. Bekey, MIT Press
Moran, Richard (1997) Self Knowledge: Discovery, Resolution, and Undoing. European Journal of Philosophy, 5:2 pp141-161, Blackwell Publishers
Shaver, Robert (2010) Thomson’s Trolley Switch. Journal of Ethics and Social Philosophy, Vol. 5, Issue 2.
Scheffler, Samuel (1985) Agent-Centred Restrictions, Rationality, and the Virtues. Mind, New Series, Vol. 94, No. 375, pp. 409-419, Oxford University Press
Liao, S. Matthew (2012) Intentions and Moral Permissibility: The Case of Acting Permissibly with Bad Intentions. Law and Philosophy 31(6): 703-724
Web Articles and Blog Posts
Clients: Hercules and Leo, Nonhuman Rights Project. Retrieved from https://www.nonhumanrights.org/hercules-leo/
Choplin, Lauren (2015) Transcript of 05/27/15 Hearing on Hercules and Leo’s habeas corpus plea. Nonhuman Rights Blog, Retrieved from https://www.nonhumanrights.org/blog/transcript-of-the-hearing-re-hercules-and-leo-at-the-supreme-court/
Fagan, Andrew (2017) Human Rights. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Fieser, James (2017) Ethics. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Kim, Shin (2017) Moral Realism. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Kind, Amy (2017) Qualia. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Murphy, Peter (2017) Coherentism in Epistemology. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Murray, Dale (2017) Nozick, Robert: Political Philosophy. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Nathanson, Stephen (2017) Utilitarianism, Act and Rule. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Poston, Ted (2017) Foundationalism. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Richardson, Henry S. (2017) Rawls, John. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Schroeder, Doris (2017) Evolutionary Ethics. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Timpe, Kevin (2017) Free Will. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Troxell, Mary (2017) Schopenhauer, Arthur. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Weijers, Dan (2017) Hedonism. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Weisberg, Josh (2017) The Hard Problem of Consciousness. The Internet Encyclopedia of Philosophy ISSN 2161-0002 http://www.iep.utm.edu/
Alexander, Larry and Moore, Michael, "Deontological Ethics", The Stanford Encyclopedia of Philosophy (Winter 2016 Edition), Edward N. Zalta (ed.), https://plato.stanford.edu/archives/win2016/entries/ethics-deontological/
Arrhenius, Gustaf, Ryberg, Jesper and Tännsjö, Torbjörn, "The Repugnant Conclusion", The Stanford Encyclopedia of Philosophy (Spring 2017 Edition), Edward N. Zalta (ed.), https://plato.stanford.edu/archives/spr2017/entries/repugnant-conclusion/
Cole, David, "The Chinese Room Argument", The Stanford Encyclopedia of Philosophy (Winter 2015 Edition), Edward N. Zalta (ed.), https://plato.stanford.edu/archives/win2015/entries/chinese-room/
Dancy, Jonathan, "Moral Particularism", The Stanford Encyclopedia of Philosophy (Winter 2017 Edition), Edward N. Zalta (ed.), https://plato.stanford.edu/archives/win2017/entries/moral-particularism/
Freeman, Samuel, "Original Position", The Stanford Encyclopedia of Philosophy (Winter 2016 Edition), Edward N. Zalta (ed.), https://plato.stanford.edu/archives/win2016/entries/original-position/
Gertler, Brie, "Self-Knowledge", The Stanford Encyclopedia of Philosophy (Fall 2017 Edition), Edward N. Zalta (ed.), https://plato.stanford.edu/archives/fall2017/entries/self-knowledge/
Gowans, Chris, "Moral Relativism", The Stanford Encyclopedia of Philosophy (Summer 2018 Edition), Edward N. Zalta (ed.), https://plato.stanford.edu/archives/sum2018/entries/moral-relativism/
Hursthouse, Rosalind and Pettigrove, Glen, "Virtue Ethics", The Stanford Encyclopedia of Philosophy (Winter 2016 Edition), Edward N. Zalta (ed.), https://plato.stanford.edu/archives/win2016/entries/ethics-virtue/
Johnson, Robert and Cureton, Adam, "Kant's Moral Philosophy", The Stanford Encyclopedia of Philosophy (Spring 2018 Edition), Edward N. Zalta (ed.), https://plato.stanford.edu/archives/spr2018/entries/kant-moral/
McIntyre, Alison, "Doctrine of Double Effect", The Stanford Encyclopedia of Philosophy (Winter 2014 Edition), Edward N. Zalta (ed.), https://plato.stanford.edu/archives/win2014/entries/double-effect/
Sinnott-Armstrong, Walter, "Consequentialism", The Stanford Encyclopedia of Philosophy (Winter 2015 Edition), Edward N. Zalta (ed.), https://plato.stanford.edu/archives/win2015/entries/consequentialism/
