An artificial intelligence that is smarter than humans is one of the favorite subjects in science fiction. There are researchers who claim that such an AI is technically impossible. Others believe it is inevitable. If that could be the case, humanity faces a difficult problem. Can we somehow ensure that this superintelligence is benevolent to us? Can we control it? Because if we can’t, the survival of humanity would be in its hands alone. A conception, which can please us so little that we would have to do then everything to prevent its emergence.

An international team of researchers has some bad news: in a new study, the scientists – with the participation of the Man and Machine Research Unit at the Max Planck Institute for Human Development – use theoretical calculations to show that it is not possible to control a superintelligent AI.

«A superintelligent machine that controls the world sounds like science fiction. But there are already machines today that perform certain important tasks on their own, without those who programmed them completely understanding how they learned to do it. So the question for us is whether this could become uncontrollable and dangerous for humanity at some point,» says Manuel Cebrian, head of the Digital Mobilization research group in the Man and Machine research area at the Max Planck Institute for Human Development and co-author of the study.

What techniques are available to get an AI under our control? First, we could specifically limit its resources by, for example, cutting it off from the Internet and all other technical devices. However, that would also reduce the usefulness of the AI. It would not be able to solve the major problems facing humanity in this way. The second option: we motivate the AI to pursue only goals that are in the interest of humanity, for example with the help of ethical rules (such as Asimov’s Robot Laws). But that doesn’t work, as the researchers show.

In their study, the team designed a theoretical algorithm that ensures that a superintelligent AI will not harm humanity under any circumstances. This algorithm first simulates the AI’s behavior and stops it if it deems its actions harmful. This sounds sensible, but there is a problem: According to the current state of computer science, it is impossible to program such an algorithm.

«Breaking the problem down to simple ground rules from theoretical computer science, it becomes apparent that an algorithm that would command an AI not to destroy the world could inadvertently bring its own processes to a halt. One would then not know if the algorithm was still analyzing the threat or if it had stopped trying to contain the harmful AI. That makes that algorithm virtually useless,» says Iyad Rahwan, director of the Man and Machine research area.

So we’d better prevent super-intelligent AI from emerging! The problem, however, has a second level: According to current knowledge, it is also impossible to calculate whether a machine possesses intelligence superior to that of humans. This means that we would have to stop any AI development immediately – otherwise we march ignorantly beyond the point where it could still be stopped. Since that won’t happen, we will undoubtedly have to live with such superintelligent AIs at some point. Unless the researchers who doubt their existence are right.