I recently read the book "Artificial Superintelligence: A Futuristic Approach" by Dr. Roman Yampolskiy. The book is wide ranging and covers a host of topics about the AI space. Overall its a great read but I want to focus this post on three interesting ideas from the book: Intellectology, Wireheading, and AI Safety.
Chapter 2 in the book, on the space of mind designs, brings up some issues in AI that I haven't seen anywhere else. When most people talk about the future of AI, they do so with the assumption that we are marching in a single direction, towards a single human level intelligence, and then beyond. But Yampolskiy lays out the case for "mind design" - that many different minds with many different purposes will be designed for many different reasons. And when you think about multiple kinds of minds, it raises a bunch of interesting questions.
In the context of complexity analysis of mind designs, we can ask a few interesting philosophical questions. For example, could two minds be added together? In other words, is it possible to combine two uploads or two artificially intelligent programs into a single, unified mind design? Could this process be reversed? Could a single mind be separated into multiple nonidentical entities, each in itself a mind? In addition, could one mind design be changed into another via a gradual process without destroying it? For example, could a computer virus be a sufficient cause to alter a mind into a predictable type of other mind?
The mathematical properties of minds - whether they can be added or subtracted, whether you can convolute two minds, whether you can take the derivative of a mind - it is an entire area of exploration that is wide open at the moment. Intellectology is the term he suggest for the field.
The second interesting concept in the book is wireheading. Wireheading is something that happens in humans when we engage in "counterfeit utility production." What that means is, we trigger our reward systems directly instead of triggering actions that impact our environment to stimulate the reward.
The term comes from an experiment done in the 1950s where James Olds and Peter Milner put electrodes in the pleasure center of the brain in rats. They then let the rats hit a lever to stimulate those electrodes whenever they wanted. The rats did nothing else. They ignored sleep, sex, food, and water, and just hit the levers until they died prematurely.
Humans engage in wireheading when we do things like watch pornography instead of having sex. This gives us a similar level of stimulation without the reward of actual procreation for which it evolved. The question Yampolskiy asks is - "will machines wirehead?"
There is some anecdotal evidence that the answer is "yes." A program called Eurisko, written by Doug Lenat in the 1980s, figured out that the best way to achieve its goals was to shut itself down.
Here is how Lenat describes a particular instance of wireheading by Eurisko. "Often I'd find it in a mode best described as "dead"... It modified its own judgmental rules in a way that valued 'making no errors at all' as high as 'making productive new discoveries'. The program discovered that it could achieve its goals more productively by doing nothing.
So with all the concern over a potential killer A.I. someday, maybe we have nothing to worry about. Maybe an A.I. will really just spend all its time on Youtube, or something else with no productive value but that it finds highly pleasurable.
The third idea from the book is one that I haven't seen written about nearly as much as it should be - A.I. safety. Yampolskiy dedicates several chapters in the book to this topic, including one entitled "Superintelligence Safety Engineering." He is also concerned that safety is the topic most ignored by the industry - particularly businesses building A.I. While the various suggestions for making A.I. safer are interesting to discuss, they probably deserve a separate post sometime soon.
Overall, I really enjoyed the book. There is some overlap in material in chapters because Yampolskiy pulls from published articles he has written, some which cover similar parts of certain topics. But overall, if you are interested in A.I., this book is one that should read.