(knocking in background)
- There are many situations in war
that are gray
and that's a real challenge
when it comes to autonomous weapons.
How would a robot know the difference
between what's legal and what's right?
How would it know how to tell the difference?
Is it acceptable for a machine to
make life and death decisions in war?
And if we answer it,
that may help us give us a guide
for how we use this technology going forward
in ways that might make warfare more humane,
but also don't lose our humanity in the process.
(high intensity music)
When people worry about autonomous weapons,
one of the fears would be robots running amok
and killing civilians.
(weapon blasts in background)
These are things you see at "Terminator" movies.
No one's building that.
That's the good news.
We're talking about weapons that once launched
and programmed and built by a human
would then make a decision on their own about
which specific targets to attack.
(weapon blasts in background)
One of the arguments for autonomous weapons is that
just like some day self-driving cars
can reduce accidents on roads
perhaps machines could do the same in war.
An autonomous weapons could avoid civilian casualties,
making war more precise and more humane.
In World War II,
nations bombed entire cities
and killed hundreds of thousands of civilians.
One of the unfortunate realities of the technology
at the time was bombing was not very precise.
Today, advanced militaries can actually
place a bomb precisely where they want to.
(suspenseful music)
I was involved in wrestling with a lot of these challenges
when I worked in the Pentagon as a Policy Analyst
in the office of the Secretary of Defense.
Where I worked on various emerging technologies
including autonomy and robotics.
There are some ways that we have experienced using autonomy,
things like homing images or automated defensive systems,
that the military has already used for decades.
Swarms are the next step in military robotics.
With the humans no longer manually controlling robots
or directing them where to go
but instead telling the swarm what to do
ordering the swarm to say,
patrol an area or conduct reconnaissance.
If people were experimenting with what this may look,
it can help with dramatically change
military tactics and war fighting.
How do you fight with a swarm?
What are the commands to give a swarm?
How do swarms fight each other?
What are the best tactics for swarm warfare?
The reality is that we can test autonomous systems
all that we want in training environments and simulations
and we can never perfectly simulate the chaos of war.
What if anything requires uniquely human judgment in war
and why?
Are there some things that we don't want machines to do at all?
I think that's a really important question to ask.
I saw things first hand in war that
I don't think machines could deal with.
Situations that I faced that I think are too complex that
require human judgment and moral understanding.
There was an incident early in the war in Afghanistan,
where I was part of an Army Ranger Sniper team
and on one mission we infiltrated at night
and when sun came up we were pretty exposed.
Now, what we expected to happen was that
some fighters would come to attack us,
and we were ready for that.
What we didn't expect was
they sent a little girl to scout our position.
(suspenseful music)
She walked a long, slow circle around us
and watched us for a while.
And we heard the chirping of what we later realized was
a radio that she was carrying with her,
reporting back information about us.
Basically she left
and not long after some Taliban fighters did come.
Then we took care of them.
(weapons blast in background)
But later we were talking about what would we do
in similar situation like that.
If we came across someone who we didn't know or
if they were civilian,
if they were scouting for the enemy,
what would we do?
Something that never came up in conversation was
the idea of shooting this little girl.
The laws of war don't set an age for combatants.
Your status as a combatant is based on your behavior.
And by scouting for the enemy
she was directly participating in hostilities.
If you programmed a robot to
comply perfectly with the laws of war
it would have killed this little girl.
(weapon blasts in background)
And I think that's wrong.
I think that would have been morally wrong.
If we had a war,
and no one felt bad about the fact that
we were killing other human beings,
what would that say about us?
What would that say about us if
no one slept uneasy at night afterwards?
These moments of humanity seem important.
They seem like something we wouldn't wanna give up.
It's an opportunity for humans to exercise empathy
and recognize the humanity of the other side.
War presents lots of complex
moral challenges that people face,
and I don't really think that any machines that we see today
or in the foreseeable future
will be able to deal with those challenges.
Not because the machines cannot make those decisions
but because they shouldn't.
And I think there's a really important role
for human judgment in making these moral decisions at war.
And as we move forward to technology
we don't wanna give that up.
We wanna keep humans in charge of making these decisions.
- Thanks for watching this episode of Uprising.
Please like and subscribe.
It really helps us grow our channel
and tell more stories about people who are
thinking differently and changing the world.
Check out Paul Scarre's book, Army of None.