“Worlds governed by artificial intelligence
often learned a hard lesson:
Logic Doesn’t Care.”
Yin-man Wei, “This Present Darkness:
A History of the Interregnum”, CY 11956
Andromeda
Giving a course in media and computing, I have the pleasure of discussing a lot of interesting topics in class. One topic was emotional machines. We had a look at Norman’s “Emotional Design“, in which he discusses Asimov’s Four Laws of Robotics:
Asimov’s Four Laws of Robotics
Zeroth Law: A robot may not injure humanity, or, through inaction, allow humanity to come to harm.
First law: A robot may not injure a human being, or, through inaction, allow a human being to come to harm, unless this would violate the Zeroth Law of Robotics.
Second Law: A robot must obey orders given it by human beings, except where such orders would conflict with the Zeroth or First Law.
Third Law: A robot must protect its own existence as long as such protection does not conflict with the Zeroth, First, or Second Law.
While Norman’s book had a couple of interesting points about emotional design, I think it underestimated the dangers of the zeroth law. Regarding the Zeroth Law, Norman writes:
The Zeroth Law — that “a robot may not injure humanity, or, through inaction, allow humanity to come to harm,” is beyond current capability, for much the same reasons that Asimov did not need this law in his early stories: to determine just when an action — or lack of action — will harm all humanity is truly sophisticated, probably beyond the abilities of most people.
“Probably beyond the abilities of most people” — are you kidding me? Deciding when an action, or lack of action, injures humanity itself is not beyond the abilities of most people but of all people. And hopefully, it always will be. This decision goes to the heart of what values our society wants to live by, wants to strive for, wants to reinforce — and which it does not want to live, try to avoid, and punish. It’s a fundamental question that is completely in the realm of values, where there are no easy absolutist, multiplist, or evaluativist answers.
This doesn’t mean that any society goes, far from it. I agree with Isaiah Berlin:
“I am not a relativist; I do not say ‘I like my coffee with milk and you like it without; I am in favor of kindness and you prefer concentration camps’ — each of us with his own values, which cannot be overcome or integrated. This I believe to be false.”
There is a better and worse when it comes to values. For example, I think it’s shameful that Western societies look on when a self-proclaimed “government” throws homosexuals from buildings. And yeah, there are such things as “just wars”, even though war should be avoided as much as possible. But sometimes, it isn’t possible anymore, because the costs are too high (stopping Nazi-Germany in World War II was such a case of a just war — when looking on was no longer possible).
But beyond cases that come with a high body count, deciding what is best for mankind is an impossible question to answer. “I, Robot” (the Will Smith movie) shows what can go wrong when the computer decides to protect mankind by treating all people like animals in a zoo. It protects them from harming themselves and prevents mankind from annihilating themselves in a war. But that’s hardly a good life, yet even with a Zeroth law (missing in the movie), it wouldn’t have stopped the computer. It could still have decided that this was the best way for mankind to live. Another solution (more the short story version) is the machines manipulating mankind in thinking they are free, while they are essentially controlled by the machines. Not that impossible — you’d be surprised what is possible with, well conditioning, nudging, making sure you observe certain things (observation learning), giving people specific situations to work in, facilitating “happy accidents” and much more. A zoo where you cannot see the bars. That’s even worse. To quote Johann Wolfgang von Goethe: “None are more hopelessly enslaved than those who falsely believe they are free.”
And these aren’t the only wrong solutions. Whatever someone will come up with how “mankind” should be, others will disagree. What someone will see as beneficial, others will see as harmful. What harms mankind — that addresses the core of what it means to be human, and that’s something mankind hasn’t figured out for itself. So how could a computer do it? And why should we let a computer make that decision?
No, hell no, that’s something a computer shouldn’t touch — ever. That’s something computers cannot decide, should not decide, but what mankind itself — the men and women it consists of — have to determine. Mostly by living their lives as they think it’s best for mankind.
P.S.: The other laws aren’t that much better. It’s easy to demand things you don’t have to do yourself, and with this laws, no human being could fulfill them (hey, you could have used some of your spare time to save a couple of lives, and that pizza you ate and the booze you drank, totally doing harm to your body). Looking at these laws and the idea of having robots — I can’t help but compare them either to Jinns or to angels. Jinns because they seem to be seen as wish-fulfillers (esp. when it comes to doing the washing, or more accurately, to ironing and folding the clothes). However, most likely, they will do what people say, not what they mean, so — like with a Jinn — expect a lot of literal interpretation of wishes that will do more harm than good. And Angels, well, mostly because they could have the potential to be “better” than humans — more rational, never failing, never faltering. However, like Angels, they would also be sterile (in the sense of not sharing the dirty bits of life) and … well, too perfect. Too perfect little servants and guards. Frankly, when it comes to robots, I’d like them to start small. Something that doesn’t do much damage when things get wrong — and they will go wrong. And yeah, they should stay away from high-level judgments about the mankind, or even, what is good for the human they work for.
Update (2015-11-22): Some slight clarifications.
