It looks like while general public is dreaming about communist utopia ruled by a benevolent Artificial Superintelligence, the corporations are racing toward creating ASI tuned for their own goals and lobbying AI regulations to impede development of competitors.
I kind of envision an alternative to all of that – to become superintelligence yourself. And the future has two distinctive possibilities: either absolute dictatorship of a singular superintelligence, or an armed balance of multiple superintelligences in ancap model.
Imagine an ultimate picture of you with your brain interfaced with some high-tech increasing your intellectual capabilities and data throughput. Such intellectual capabilities that you could contemplate quantum gravity theories or analize the genome of your cat. Such data throughput that you could manage in real time swarms of combat drones in the battlefield like it were your own fingers. Or you could manage all cyber-servants of your international restaurant chain as if you were the every one of them.
I've mentioned military application for a reason. New technologies give an edge in the power struggle, and thus attract power players. Intelligent people usually avoid direct participation in conflicts. They invent weapons but let harsh military men do the dirty job. This will be not the case anymore.
Such technology eliminates the need for specialization and labor division. If you have not enough resources to solve some problem and need the help of your neighbour, you may as well devour his resources and do the job by youself. This sounds pretty dangerous, isn't it? Like in the "Highlander" movie, leading to the survival of a singular, most aggressive superintelligence.
The hope for humanity is in simultaneous emergence of multiple superintelligences that will keep an armed balance of power. They will be smart enough to understand that peace is the best strategy and will be uniting to fight off aggressors. Just like in primitive society, but without specialization being the main drive for cooperation.
More on this in my article on LessWrong: Become a Superintelligence Yourself
Posted by graniar