Speaker 1: Today, we're going to be speaking with Eliezer Ude Koski, who's the founder
and senior research fellow of the Machine Intelligence Research Institute, which is
an organization dedicated to ensuring that smarter than human AI has a has a positive
impact on the world.
Ali has a really appreciate your time and insights today.
Speaker 2: Thank you very much.
Speaker 1: So, I mean, to start with, many of the discussions around AI, particularly
now with Chat GPT four and the questions of what will this look like in six months and
12 months and 24 months?
What will the impact be on jobs?
What?
What about if it turns evil or it turns against humanity or whatever?
Are the conversations about A.I.
even being framed in the right parameters as far as you see?
Speaker 2: Well, we try to frame it correctly as we see it ourselves and see whether we
have the chance.
I think there is widespread agreement that Chhatrapati and GPT four are relatively unlikely
to end the world, you know, either exterminate humanity or have a very large impact outside
of that.
So so anyone who opens it by with the frame of like like look at just GPT four, look at
only chat.
CBT is probably there to convince you it's not going to have a large impact.
Speaker 1: And is it wrong to limit the conversation only to GPG for.
Speaker 2: Obviously the future exists.
You know, tomorrow is going to be a different day from today.
In five years.
It's not going to be this year.
Speaker 1: What is the physical mechanism?
And a bunch of people wrote some version of this question to me when we said we'd be having
you on, when when there's this idea, the scary idea of an AI going beyond a computer screen
and starting to function or manipulate the real world.
A lot of people were saying to me, Can you ask what is the mechanism through which that
would happen?
Would it be by controlling systems like traffic lights and airplanes or like what?
Because at the end of the day, we're talking about software.
If something like that were to happen, what's the physical process?
Speaker 2: I mean, this is a deep question.
This is going to take a while to answer.
Speaker 1: Well, okay.
Speaker 2: All right.
So, first of all, even GPT four is already at the level where if you ask it to hire a
TaskRabbit and think out loud about how to hire a TaskRabbit in order to bypass CAPTCHAs
that are meant to restrict systems for human use only, it's already advanced enough that
when the person that was trying to hire as a TaskRabbit said like So are you a robot
that you can't read the CAPTCHAs while.
Deep tea for thought out loud.
Like I should conceal that for the fact that I'm a robot.
I should make up a reason that I can't solve the capture.
I'm going to name Channel told the task Rabbit.
No, I have a visual impairment.
Speaker 1: Right.
Speaker 2: So it's already like you might say that it's already like able to use task
rabbits as fingers in the real world, and it's already understands humans out in the
real world well enough to know that it shouldn't just tell people to robot if it wants to get
its job done.
So.
So that's today's systems.
Speaker 1: Right.
Speaker 2: That's relatively straightforward.
I can tell you about it because it already happened.
Predicting the future is harder.
Predicting what's something smarter than you could do is fundamentally harder.
If I was playing chess against Garry Kasparov, the past world champion who lost to Deep Blue,
if I could predict exactly where Garry Kasparov would move against me on the chessboard, I
could play chess as least as at least as well as Kasparov by just moving whenever I predicted
Kasparov would move.
So something really actually smarter than you.
You cannot predict unless it's a very narrow game.
Like tic tac toe can maybe predict where somebody will move against you in tic tac toe, even
if they're smarter, Right?
The real world is much more complicated than tic tac toe.
So the basic question as to how an AI gets out of the computer.
The basic answer is, if I knew exactly how it would do that, I would have to be as smart
as the A.I..
With that fundamental obstacle in mind.
The more you sort of look into this stuff and study where the current technological
roadblocks are, the better the guess you can take at how a smarter opponent might move
against you.
Or like setting lower bounds.
Like, what can somebody do if they're able to solve technological problems that we understand
well enough to know a smarter mind could solve them?
Hmm.
So if you're just coming out this and you don't know anything about exotic technologies
that haven't been developed yet, you might be like, Yeah, we'll use TaskRabbit.
It'll pay humans that don't know it's an AI.
It'll blackmail humans.
Still no reason for it to tell it that it's an AI.
It'll say it's an AI to, you know, online wacky cultists who believe that they should
destroy everything.
And those are its human hands.
You'll ask yourself, what could I do with some human hands of five or an AI?
You might imagine that it gets a hold of some GPUs and tries to build a backup for itself
someplace that humans don't know about.
You might imagine that it writes the next version of itself and makes it even smarter.
Mm hmm.
You know, you're trying to already imagine what if it's smart?
So saying that doesn't help a lot in some ways.
But, you know, maybe you can make itself smaller, more efficient, maybe can back itself up in
multiple places in computers where you would not expect that there was an AI on board.
Maybe it can find unknown security holes in a cell phone In many varieties of cell phones.
Start listening to human conversations.
Get more blackmail material.
Maybe I can pretend to be human and, you know, make online friends with a bunch of voters.
Speaker 1: Right?
Speaker 2: Persuade them to vote for candidates that seem outwardly nice, but which it in
fact has under fairly detailed blackmailed control.
And this is sort of like without any of the difficult to understand stuff.
Speaker 1: Right?
That's based on mechanisms and social cultural realities and technologies that exist and
that we understand which which there's an entire other category of things that we simply
can't conceive of yet.
Speaker 2: Right.
So it's like, suppose you ask the 11th century, like a portal opens to the 21st century in
the middle of, say, modern Russia, you know, like what are your concerns about fighting
somebody from a thousand years in the future?
They might be like, well, you know what?
If it's a wealthier country?
What if they have more knights?
What if more of their knights are armored?
Speaker 1: Right.
Speaker 2: And, you know, getting they're just not going to get from their nuclear weapons
if they imagine that Russia of the future has irresistibly powerful sorcery.
They'll get close to the truth.
It's not actually irresistible and it's not actually sorcery.
Speaker 1: Right.
Speaker 2: But, you know, you have to reach pretty far, too, to expect that among the
resources they have are nuclear weapons.
And Russia's not actually going to bother with the nuclear weapons they don't need.
It can send a tank rolling through a 11th century battlefield.
And, you know, maybe if they see it coming, they can dig a pit trap.
But otherwise, all the arrows and lances are going to bounce off it all day long.
It just rolls over the horses.
Speaker 1: Right, Right.
It's so we're talking about orders of magnitude difference.
So, you know, if we imagine some sort of spectrum of opinion on this issue, you do have some
folks who say this is, as you alluded to, the most dangerous thing we can imagine or
can't imagine must be stopped immediately in its tracks with either legislation or regulation
or whatever the case may be.
On the other side, I read some interesting op eds that basically say this is one of the
best things we can imagine.
This is going to ten x the productivity of every normal worker.
It's going to do so.
Many of the things that people said in the thirties were going to happen where everybody
would only be working 12 hours a week and have everything they need.
Right.
That didn't happen.
We know how that worked out.
But there's the very pro side and then there's sort of like what I think maybe Neil Postman's
view would have been if he were writing about it today, which was his view on prior new
technologies, which was spending too much time simply on trying to block them, is not
the most useful thing.
We're better off harnessing the positive and regulating the potential risks, and that would
maybe be an in the middle sort of view.
What do you think makes the most sense based on what we know right now as an approach?
Speaker 2: Well, I think that talking about it like it's a normal technology and not something
smarter than you is basically misguided and have the wrong conversation.
Speaker 1: So it's not even in the right category of the stuff Postman talked about.
Speaker 2: It's not electricity, right?
It's not nuclear power.
It's not nuclear weapons.
It's not even a super virus.
It's something that has its own plans.
You don't get to just plan how to use it.
It is planning how to use itself.
Speaker 1: So a totally different paradigm needs to be applied here.
Speaker 2: I mean, imagine if somebody was like, Well, we're about to contact these aliens
with much more advanced technology that think faster than us, that are smarter than us,
that have been around the stars for a while.
They're landing.
We're not quite sure exactly when.
We're not sure how fast their spaceships are.
You know, it might be two years, it might be 30 years.
And somebody is like, well, you know, this is just like electricity.
Speaker 1: Right, Right.
Let's use the cool stuff about their technology, but make sure they don't do anything bad.
Right.
It would be silly to say it in that way.
Speaker 2: Precisely.
Yeah.
Speaker 1: So.
So then what do you what do you think needs to be done at this point?
Speaker 2: I mean, I one of the crux of the issue is can you do nice things with it?
And the like.
Grim, dark message that I that I am burying is that we don't know how to do that and we're
not likely to figure it out in time.
And there's a very large gap between where we are and understanding what goes on inside
the frontier eyes and understanding and being able to shape them in detail in a way where
they go on wanting to be nice or even in a certain sense, wanting to be, you know, non-gender
to like just do particular tasks and do them without lots of side effects in like the most
normal possible way that they can, that they can accomplish those tasks, like even building
that sort of limited thing, never mind something that is really friends with you is I think,
beyond the range of what we're going to figure out in the foreseeable future.
And I do think we are storming directly ahead on capabilities, and that looks like a giant
disaster in the making.
So, yeah, I side with well, I think my my basic factual point of view is if you guess
if that is that if we do not somehow avoid doing because we're all going to die.
And my corresponding policy view is that we should not do this even if that's very, really
quite hard and requires us to do some unusual things.
Speaker 1: So in terms of what to do, I mean, I guess if everybody came around to your view,
you would think anybody developing these technologies would just give it up right on their own.
They would say, Wow, Eliezer is right.
I share that concern.
The right thing to do is for me to stop working on this.
Assuming that that doesn't happen.
What is it that you would like to see?
Should this type of research and development be made against the law?
Speaker 2: Yeah, basically, I think that we should track all the GPOs, have international
arrangements for all of the AI, all of the training tech to end up in only monitored,
supervised licensed data centers and allied countries.
And if your any time you have any and just like not permit training runs more powerful
than GPT four, if we're not going to do that, then we should monitor all the training runs
larger than GPT four or rather like GPT four sized and keep the model weights inside.
Only the licensed regulated data centers and track who is running them.
Store the output someplace.
You can look at the outputs with the warrant.
A bunch of that technique like that whole line of reasoning is not that regulating the
stuff will protect you from a superintelligence because it will not.
Mm hmm.
That's more in the hopes that people change their minds later, maybe after some major
disaster that doesn't kill everyone.
And in that case, the technology that you would need to sue somebody who, you know,
killed a dozen people at a hospital or cost $100 million worth of damage or whatever kicked
out, you need to know who did that and sue them and not just have all the AI development
go to places where they couldn't be sued is the same technology that humanity would need
to have a off switch that if they were you don't like press the off switch to deal with
the superintelligence.
The superintelligence is not let you know that you need to tell the off switch, that
you need to press the off switch until you are already dead.
But if you get it, if we're lucky enough to get warning signs, then the civilizational
infrastructure to have a pause button is the same as the civilized civilizational infrastructure
to sue people who do small amounts of damage like small as in survivable as in there were
survivors.
Speaker 1: As someone who very clearly you've expressed your very serious concerns with
this, do you also see this technology as something that could and this doesn't mean that it would
or that it would be a good trade for the things that you're talking about?
But do you also see this as a technology that could, for example, do analysis of the human
genome such that it would accelerate us in terms of our ability to cure or prevent disease,
the likes of which could take who knows how long without such technology or when it comes
to energy or whatever.
Do you also see that side and or does it not matter because the downside is so huge?
Speaker 2: I mean, if we knew how to build an AI that did exactly what we wanted, we
could thereby spread out across the galaxy is turning them into our cities full of sapient
sentient beings, you know, enjoying themselves and caring about each other and generally
living happily ever after until the last of the entropy runs out.
You know, that's always been the dream.
Yeah.
We don't know how to do that.
It's physically possible, but the art and technique to do it is beyond us in the present
time, and that's going to not be gained in the next five years.
And I'm not sure how you would task a government bureaucracy to recognize it, even if somebody
came up with it in 40 years.
It's like there's a there's a.
There's a sort of basic question about whether our civilization is smart enough to do something
correctly on the first try at all.
The way science usually works is that you have a bunch of loony eyed optimists with
wacky theories who storm ahead on their basic research problem.
And they are wrong.
And, you know, they go back to the drawing board and they're wrong again.
And, you know, like the next generation is a bit more cynical about how hard the problem
is and eventually people work it out.
And if we were allowed to do that with superintelligence, I would have much, much, much less fear of
the outcome.
Still still some fear because, you know, you're playing with pretty high stakes there.
You might have somebody who like, you know, the people who end up figuring out how to
line it might misuse it.
It still be that major threat, But it wouldn't be like the automatic extinction scenario
if we had the textbook from 100 years in the future that contains all the simple ideas
that actually work, right?
That takes so long to identify and practice the rules instead of sitting wise.
For those of us who've been following A.I.
for the last couple of years and know what I'm talking about there.
But, you know, like, there's pieces of there's there's all kinds of places in A.I.
where people try to do things using complicated techniques, and they didn't work.
And 20 years later, the camp with the simple technique that actually works.
And if you have the textbook from the future with all of the simple things that actually
work in practice for alignment is probably not hard.
And you can get all the goodies, but we don't have that textbook.
And the first problem is that, you know, like getting the sample correct on the first.
The first problem is the amount of time it takes to get that where capabilities are storming
ahead because you can tell whether things are working or not on capabilities.
Whereas with alignment, you've got to, you know, know that at the point where it's much
smarter than you, it's already aligned, you don't get retried past that point.
So it's more like launching a space probe that has to land on Mars correctly the first
time than it is with like building a car, watching it break down and tinkering with
the car.
Speaker 1: And if it doesn't, it might break everything so you don't get to try again.
Speaker 2: Yeah, like the entire human species is is is packed on board the Mars probe and
the first rocket that goes high enough has to land on Mars.
There aren't even really good analogies for humanity.
Has not faced an issue like this before, which is why we're still around.
And I think looking at this, that this is just like clearly beyond the reach of our
present civilization.
It's not close.
It's outside the range of things that you could reasonably that you could tell a reasonable
worry about people doing successfully.
And that's why we we need to back off.
Or rather, I predict that if we don't back off, we die.
And I wish we would back off.
I don't predict that we will.
Speaker 1: Well, certainly there's no way to wrap up such a dark vision in any way that
is going going to be satisfactory or calming to the audience, I think.
But that being said, allies are your Koski is taking this very seriously.
As the founder and senior research fellow of the Machine Intelligence Research Institute.
And I really do appreciate your time and insights, even if they are not optimistic on this issue.
Speaker 2: Thank you.
Thanks for having me on.