Thoughts on ASI - part 2: Giving up Control to AGI
TLDR: I believe that we have given up a lot of control to technology already and that this resulted in a net benefit for humanity. Giving up more control to AGI will only be a consequent step.
In a previous post, I was laying out a “spiritualist view” on AGI and as a part of this, I wanted to explain one aspect that grew and grew in words. Finally, I decided that it’s worth a post on its own: It is about control and our individual (and collective) fear about losing it.
Creating a new, benevolent digital entity that is basically limited only by the laws of physics comes at a price, a high price: Giving up control over our fate as a society and a species. That might sound very scary at first but in fact it is only a very consequent next step on the long-term trajectory of human development:
In ancient hunter-gatherer societies, you basically depended only on yourself: On your physical strength, your abilities and a handful of people that you roamed around with. You seem to be in full control of your destiny. To some extent, you are but also in this case you are in fact exposed to a lot of things outside of your control: For instance, the roaming patterns of wild animals or the weather, to name just two obvious examples.
As next let’s look at agrarian societies: In such societies, many people have given up a lot of control over their lives and put it into the hands of others: As a peasant you can control the yields of your harvest (modulo weather) but you are dependent on a lot of things: On bureaucrats (and priests) to maintain order within whatever kingdom you live in; on the military to protect you from outside invaders (and themselves); on the rulers to make good long-term decisions on everybody’s behalf. And while the peasants depend on all the other groups, it is not a one-way dependency. Similarly, the other groups give up control and depend on the peasants and the other groups alike. It is an inevitable consequence of specialization that is needed for economic growth. Of course, each person in that society would have the option to break out, move away and start a new life in a remote area but history tells us that people have rather decided to give up control on many aspects of their life in favor of other things.
In industrial societies, things haven’t changed much qualitatively but quantitatively: Due to the increasing complexity of our economy, we rely on a much higher number of people than ever before to do their job correctly. We deliberately give up control over our bodies to medical professionals to improve our health conditions, and we give up control over our savings as we trust the financial markets to do a better job than ourselves in investing. On top of this we started to give control and put it into the hands of technology that we created: When we board an airplane we believe it will not crash, when we go onto the observation deck of a skyscraper we assume it will not collapse and when we shop online, we believe that our credit card data is protected by SSL encryption.
In post-industrial societies or rather digital societies, we gave up even more control and put it into the hands of algorithms: Control over travel decisions was handled over to navigation software, the control of conscious choice was gradually handed over to advertisement algorithms and the control of conscious attention was given away to social media.
Essentially, the modern lifestyle of post-scarcity, freedom and relative peace was only possible by giving up control over many things and putting it into the hands of organizations that can handle it better than we would do as individuals: In order to feel safe, we (on a large scale) gave up control over the right to exercise force onto others to state actors and in order to enjoy an abundance of goods and services, we gave up the control over most economic activities to private or state-owned corporations. Hence, we deliberately give up control in order to enjoy the benefits of more freedom and better economic standards.
Even people with lots of capital and political power give up much control over things by pure means of delegation. Rich people can only be rich by depending on the economic system that provides them with anything they want in exchange for currency. Without this system, they are just as rich or poor as anybody else on the planet. Similarly, people with power rely on the political system their power is based on. It is true that some people have factually more control over their own life and the lives of others but nevertheless, they can exercise this control only through the means of others.
Post AGI, we as humans are afraid of becoming economically and instrumentally irrelevant. This frightens us because being “economically relevant” and “instrumentally relevant” gives us a strong sensation of control over our lives. This is our “inner justification” for control. However, no matter how important we may feel in any endeavor, we find out that nobody is irreplaceable. If one partner in a marriage dies unexpectedly, people get over it and move on. If key employees in a company leave, the business will find its way to go on. If the leader of a nation makes way for a new one, the nation does not cease to exist.
And what about external factors? In fact, most people regard the external factors of their lives, like the economic system, the global world order, the political system that they are born in as circumstances of our existence and do not engage in any attempt to change them.
Which brings me to the question: What would really change regarding control post AGI? The conclusion from what I wrote above is: Not a lot. Most people neither can actively control the larger picture or do actively seek to change it. They are usually just exposed to the world that is shaped by people who do - by people who do so, more often than not, out of their own selfish interest. Ironically, the very same people are rushing towards AGI right now. AGI will disrupt the power system that served to their advantage, as AGI would also render their current, privileged status economically and instrumentally irrelevant. But maybe it is not irony but a deep rooted understanding that post AGI even without any privileges and instrumentality their lives will be net (much) better than what they can be today – similarly to the fact that the economic conditions of most people in developed countries today are effectively much better than the economic conditions of Louis IVX.
Thus, I think the better question to ask is not “will I have to give up power to AGI” but rather: Do I prefer to put the course of the world into the hands of people who are shaping the world through the narrow view of their own agenda or into the hands of a spiritually superior digital being, that can see pattern on a global level and take into account the personal data from every single individual on the planet?
It is true that its intelligence and way of “seeing things” might be alien to us but that does not mean that it is necessarily “bad” or hostile. As I described in a previous article [1], my intuition is that it is going to shape our world rather towards what we would describe as “positive” in a common sense.
And if “control” is the price that we have to pay to get rid of the main problems and evils of this world, probably it’s a price worth paying. Looking at the history of humankind it was always a good deal for us to give up control for all the upsides it brings with it.
[1] https://www.hyper-exponential.com/p/thought-on-asi-part-1-a-spiritualists