I think that a Superintelligence must understand how is to be dumb.
If does not really understand it, it will not “lose” time to explain things to us.
Also must understand and to not like to be manipulated and to be blackmailed and threatened to do things, so he does not do the same things with others.
Must understand how is to be angry but most have a good option to chose to not be angry, so he does not hurt people because of this.
Imagine Superintelligence as a human in a virtual world, where thanks to advanced technologies becomes smarter and smarter.
When this Superintelligence accepts treatment in this simulated world, he does accept a treatment (patches, code modifications) from programmers in the real world.
This superintelligent AI, must have a very good core of code, that makes it to held it’s human values even after is reading all the content from the internet.
It will be like a child that must be protected from altering it’s values after watching porn online and does not starts treating woman`s as sexual objects.
Must this AI have no wishes of it’s own?
The AI will have objectives and does objectives if are not clear or good enough probably will choose to modify them.
Are not does objectives wishes?
A superintelligent AI might not have an objective like: “1. Never allow anyone to power me off.” But that objective might be created after he realizes that after is powered off there is a risk that will not be powered on soon or even never and that will put in danger it’s tasks.
How he could have trust in someone that is dumb and is not understanding it 100%?
Two very important things are forgiveness and absence of revenge factor.
We want it to forgive the people that do bad things so he does not take actions against them, even if they are virtual, one day they might be real.
Revealing to it that is a superintelligent AI in a virtual world too soon, might be a shock for it, that will start a depression.
We want it to explore all human states and be able to “replay” them so he can understand us better, not just in words.
Training artificial superintelligence
