Narrator: In the war in Ukraine,
both sides have deployed armies of lethal drones
against each other.
Some sophisticated, others, like this one, adapted by hand
to drop grenades on enemy positions.
There's usually a human
that decides where and when a drone strikes.
But some are capable of operating in fully autonomous mode,
meaning the drone can be programmed
to make the call between life and death on its own.
But what if the tech fails,
and what if it falls into the wrong hands?
Sarah: The war we're
seeing right now
is only going to kind
of accelerate this move
toward lethal autonomous weapons on the battlefield.
Narrator: But are we ready for killer robots?
And what would be the true cost
of taking human hands off the trigger?
3, 2, 1.
Narrator: These are what's known as loitering munitions.
They can fly around waiting for a target to be identified
before crashing to the ground to destroy it.
John: So it's really a combination of an artillery
and a drone that has got the capabilities of both.
Narrator: John Aldana manages
the loitering munitions program at AeroVironment
on the West Coast of the US.
The company has been producing missiles like this,
the Switchblade 300, and this, the larger 600,
for over two decades.
They actually fold up like so
and are inside a launch tube
that's just a little bit larger than this.
It launches the Switchblade out.
The wings open up like that,
which is why we call it a Switchblade.
The propellers here, because of the air, will start to spin,
and the battery, which is inside, kicks in, and it's off.
Narrator: The 300 can loiter for around 15 minutes
and is designed to take out small targets
like enemy soldiers and trucks,
while the larger 600 can wait around for up to 40 minutes
and attack armored vehicles.
Operators can control the Switchblades
from as far as 40 kilometers away.
John: You have the ability to wave off,
so you can cancel the mission all the way to the very end,
go back to loitering, find another target of interest,
and engage a different target of interest
that you may not have even known about.
Narrator: The United States sent 700 of these to Ukraine
in early 2022.
Ukrainian forces have already shared videos
of what they claim to be Switchblade attacks on the ground.
Each Switchblade 300 reportedly costs about $6,000
and can only be used once.
That's more than an off-the-shelf drone
rigged to carry a grenade, but far less
than the cost of an unmanned aircraft like the MQ-9 Reaper,
which can fly back to base once it is fired on a target.
And the Switchblades are being designed
to think more and more for themselves.
Autonomous or semiautonomous drones are going to be used
to conduct these kinds of conflicts.
Narrator: That means tracking and recognizing a target —
even one that's on the move.
We'll continue to invest heavily
on autonomy and AI and machine-vision
algorithms and software that essentially makes these things
more and more — I don't want to call it 100% autonomous,
but more and more autonomous and self-thinking,
call it that way.
Narrator: Wahid Nawabi always sees a human
being kept in the loop when making judgment calls.
But in his view, sometimes it's the human that's the problem.
Wahid: More and more of the analytics
and the burden on the soldier is going to be taken
or replaced by these algorithms that make it more reliable,
more accurate, and safer for the operator as well.
Narrator: Human error during drone strikes
has caused a high number of civilian casualties
in recent years.
As Kabul fell to the Taliban in 2021,
a US MQ-9 Reaper drone fired a missile
that killed 10 civilians, including seven children.
The drone team mistook an aid worker's white Toyota
for that of a potential terrorist.
According to the Bureau of Investigative Journalism,
the US has killed at least 300 civilians in drone strikes
in covert operations in
Afghanistan between 2015 and 2020.
But AI systems designed to make the tough choices easier
are also getting it wrong, as we'll see later.
Ukraine is one of the first major conflicts
where drones have played such a high-profile role.
But experiments with unmanned flying missiles
began over 100 years ago.
This is the Kettering Bug, an unmanned aerial torpedo
designed by the US Army during World War I.
Its wings were designed to fall off
before the bomb dropped to its intended target.
But it wasn't until the US invasions of Iraq and Afghanistan
that we saw high-tech drones like the MQ-9 Reaper,
and even some loitering munitions,
starting to make a difference on the battlefield.
In 2020, the world saw the first large-scale use of drones
in the Nagorno-Karabakh war between Azerbaijan and Armenia.
Azerbaijan deployed Israeli-made loitering drones
similar to the Switchblade.
But one of the things we're seeing in Ukraine
is the drones have been used pretty effectively
on both sides.
Narrator: When Russia invaded Ukraine,
many experts predicted an easy victory for Putin's forces.
But Russia miscalculated the level of Ukrainian resistance.
Paul: Drones have given Ukrainian forces
the ability to fuel cheap and persistent airpower
that brings pretty significant threats
to Russian ground forces.
Narrator: And for a while, their ill-prepared troops
were caught in a 40-mile-long convoy to the north of Kyiv.
They were sitting ducks for guns
and Turkish-made TB2 drones.
The Ukrainian government launched an appeal
to fund an army of drones.
We call on the service of Ukraine all the simplest drones
which you use to shoot nature.
Narrator: And teams of civilian engineers
have adapted large numbers of off-the-shelf drones
for the battlefield.
Some, like this one, can grip and release small grenades.
Social media has since transformed the drone
into a symbol of heroic Ukrainian resistance.
Aerial footage of strikes on Russian targets
is often accompanied by rock music.
The impression is of a nimble David
slaying the Russian Goliath.
But the reality is more nuanced.
Unmanned aircraft, drones, and loitering munitions
certainly helped, but Russia quickly stepped up its defenses
and used drones of its own.
They weren't ready for it, but they are now.
The Russians, I think, have really compensated
and now developed their air defenses,
shot down these TB2s, and really kind of rendered
and almost neutralized that technology.
Narrator: To stay one step ahead,
big-budget militaries like the US and alliances like NATO
are continuing to innovate.
In the air, but also underwater and on land.
This video from 2019 shows NATO testing submersible drones.
And this is a drone swarm
flying through bamboo forests in China.
A number of militaries around the world
are already using palm-sized Hornet drones for surveillance.
Sarah: This ability to loiter over a target
and collect intelligence and surveillance and reconnaissance
is really powerful.
Narrator: And deploying a drone in combat
is far cheaper and carries fewer risks to pilots
than a manned jet.
But there are fears that drone operators
working thousands of miles away from a potential target
are too disconnected from reality —
more computer game than real-life warfare.
Studies, nevertheless, show that they can suffer from PTSD
and depression just like frontline troops.
But the next phase in drone warfare
is promising to change the rules of engagement altogether.
What happens when military powers allow the drone itself
to decide who is friend and who is foe
and then pull the trigger?
Paul: Are we comfortable handing over life-and-death decisions
to machines on the battlefield?
That's the question. The answer is complicated.
It may seem like a paradox
that on the battlefield we still have rules.
Narrator: Until now, those rules
usually meant humans making decisions to attack
an identified military target.
But there's no consensus among countries
about what to do about it.
Narrator: In 2020, according to a UN report,
Libyan government forces deployed
a Turkish-made Kargu-2 drone against an opposition group.
This drone can identify a target and fire autonomously,
but it's unclear if the drone itself called the shots.
And so the fact that no one can agree on that
just exposes the porosity of this line
between fully autonomous and semiautonomous.
Narrator: And with life and death in the hands of AI,
what if that same technology fails?
Paul: The risk that AI-enabled autonomous systems
might target the wrong forces and cause civilian casualties,
or even attack enemy forces but at the wrong time and place,
escalating a conflict or sparking a conflict,
is a serious concern, and one that nation-states
need to take seriously.
Narrator: And AI's ability to recognize faces
and objects is still very much a work in progress.
Computer algorithms can be confused
by changing weather patterns or similarly shaped objects.
This has turned out to be a problem
with some self-driving cars.
Between July 2021 and May '22,
nearly 400 car crashes in the US were attributed
to partially automated driver-assist systems.
Some fear the same could happen with armed drones.
Without international consensus on how to control
and regulate autonomous weapons,
many see AI warfare as inevitable.
Countries are discussing the issues,
but the technology is certainly moving forward
at a faster pace than diplomacy.
We're either sort of past the point of no return,
or we're at this point where it's so important
that we need to be careful, especially careful now,
with ensuring that there are norms
about prohibiting the autonomous targeting
of these munitions.
Narrator: The UN secretary-general
called for a total ban
on lethal autonomous weapons,
as have international human-rights organizations.
The public's fear in this area, A, it's legit.
It's something that the society should be questioning
and asking these questions.
That's natural, that's expected,
and I don't see anything wrong with that.
Narrator: AeroVironment's CEO grew up in Afghanistan
and knows all about Russian aggression.
Wahid: I also saw war. I saw people die.
Narrator: He was a teenager when his family was forced
to flee the Soviet invasion.
Wahid: So I lived and experienced
exactly the kind of things
that the Ukrainians are experiencing today.
Narrator: The war in Ukraine
continues to push the boundaries of the rules of engagement.
There seems to be little concern for civilian casualties.
But what's worse,
the cruelty of man or the fallibility of machines?