Hollywood movies make people worry about the wrong things in terms of super intelligence.
What we should really worry about is not malice but competence, where we have machines that
are smarter than us whose goals just aren’t aligned with ours. For example, I don’t
hate ants, I don’t go out of my way to stomp an ant if I see one on the sidewalk, but if
I’m in charge of this hydroelectric dam construction and just as I’m going to flood
this valley with water I see an ant hill there, tough luck for the ants. Their goals weren’t
aligned with mine and because I’m smarter it’s going to be my goals, not the ant’s
goals, that get fulfilled. We never want to put humanity in the role of those ants.
On the other hand it doesn’t have to be bad if you solve the goal alignment problem.
Little babies tend to be in a household surrounded by human level intelligence as they’re smarter
than the babies, namely their parents. And that works out fine because the goals of the
parents are wonderfully aligned with the goals of the child’s so it’s all good. And this
is one vision that a lot of AI researchers have, the friendly AI vision that we will
succeed in not just making machines that are smarter than us, but also machines that then
learn, adopt and retain our goals as they get ever smarter.
It might sound easy to get machines to learn, adopt and retain our goals, but these are
all very tough problems. First of all, if you take a self-driving taxi and tell it in
the future to take you to the airport as fast as possible and then you get there covered
in vomit and chased by helicopters and you say, “No, no, no! That’s not what I wanted!”
and it replies, “That is exactly what you asked for,” then you’ve appreciated how
hard it is to get a machine to understand your goals, your actual goals.
A human cabdriver would have realized that you also had other goals that were unstated
because she was also a human and has all this shared reference frame, but a machine doesn’t
have that unless we explicitly teach it that. And then once the machine understands our
goals there’s a separate problem of getting them to adopt the goals. Anyone who has had
kids knows how big the difference is between making the kids understand what you want and
actually adopt your goals to do what you want.
And finally, even if you can get your kids to adopt your goals that doesn’t mean they’re
going to retain them for life. My kids are a lot less excited about Lego now than they
were when they were little, and we don’t want machines as they get ever-smarter to
gradually change their goals away from being excited about protecting us and thinking of
this thing about taking care of humanity as this little childhood thing (like Legos) that
they get bored with eventually.
If we can solve all three of these challenges, getting machines to understand our goals,
adopt them and retain them then we can create an awesome future. Because everything I love
about civilization is a product of intelligence. Then if we can use machines to amplify our
intelligence then we have this potential to solve all the problems that are stumping us
today and create a better future than we even dare to dream of.
If machines ever surpass us and can outsmart us at all tasks that’s going to be a really
big deal because intelligence is power. The reason that we humans have more power on this
planet than tigers is not because we have larger muscles or sharper claws, it’s because
we’re smarter than the tigers. And in the exact same way if machines are smarter than
us it becomes perfectly plausible for them to control us and become the rulers of this
planet and beyond. When I. J. Good made this famous analysis of how you could get an intelligence
explosion, or intelligence just kept creating greater and greater intelligence leaving us
far behind, he also mentioned that this super intelligence would be the last invention that
man need ever make. And what he meant by that, of course, was that so far the most intelligent
being on this planet that’s been doing all the inventing—it’s been us. But once we
make machines that are better than us at inventing, all future technology that we ever need can
be created by those machines if we can make sure that they do things for us that we want
and help us create an awesome future where humanity can flourish like never before.