- [Narrator] Artificial intelligence
has been threatening everyone's jobs,
including interpreters.
Inaccurate interpretation can be the difference
between a good or bad nuclear deal.
So Wired tested a speech translator called KUDO
versus human interpreters to see which better transmits all
of the necessary elements
for people to understand what they're hearing,
including emotion, intonation, completeness,
spontaneity and speed.
- [KUDO] Liars jobs.
- [Narrator] To test out if AI is really better
than a human.
Let's begin.
Test one.
- Here we go.
- [Narrator] We gave Barry and KUDO a 2020 speech
from King Felipe of Spain.
The speech is about the COVID crisis.
The subjects will focus on emotion.
- Because of the pandemic.
I parachuted into this not knowing anything
about the speech, when it was given.
In this speech, what matters more
is the emotion and the ability to connect
with the audience because he's speaking to the citizens
of Spain at a very difficult time
when many have suffered loss,
are going through a number of challenges.
This reminds us of very deep feelings.
At this time, many of you are fighting against the illness
and what it has done in your homes.
And the words are just the vehicle to get that across.
- [Narrator] Now let's evaluate the speech translator.
- [KUDO] Good evening, I address you on this Christmas Eve
when we are living truly exceptional circumstances
due to the pandemic.
- In terms of content and completeness,
the AI did very well.
- [KUDO] The virus has broken into our lives,
bringing suffering, sadness or fear,
altered our way of life.
- It struggled with word choice in a number of areas.
It referred to an overflow in some hospitals.
- [KUDO] And also overflow in some of our hospitals.
- Now, what that was referring to
is at that time when hospitals were overwhelmed,
when the king was recognizing the healthcare workers
for the work that they had done
and he recognized the tremendous physical
and emotional burden that they were carrying
and the AI translated that
as the great emotional charge and physics on their backs.
If you know Spanish, you can go back and figure out
how it is that the AI determined to use those words.
But obviously, that is borderline nonsensical.
If you look
at what the speech-to-speech translation produced,
it was by and large, quite accurate.
However, it's very difficult to get the sense
of what the king is seeking to portray
through the AI in its current state.
- [Narrator] Test two, spontaneity.
We gave Walter and KUDO a 2021 speech from former senator
and now current Colombian president, Gustavo Petro.
- The most powerful and nonproductive sectors
of this country, 20 billion pesos a year.
So I took the hour beforehand to look him up,
look at some of his videos.
Just listen to the level of language he employs.
Listen to his rhythms.
Listen to, you know, watch his body language.
- [Narrator] Now let's see how the speech translator did.
- [KUDO] You are, Mr. President-in-Office.
[Gustavo speaking in Spanish]
- [KUDO] In the year of 2019, this Congress decided to give
to the most powerful and unproductive sectors
of the country, 20 billion pesos a year.
- The particular strength of artificial intelligence
was in capturing those terms of art.
Artificial intelligence captured president-in-office
or sitting president,
[Walter speaking in Spanish]
minister of finance,
[Walter speaking in Spanish]
special drawing rights.
That's a consecrated term.
It's a term that's enshrined
that you can find in any number of glossaries.
That's the official translation, and I missed that.
Official caucuses as well
that the artificial translation rendered that very clearly
and it pulls those in and did so very well.
That's what I felt it was most effective at.
And where I felt it had the most shortcomings
was because of the pauses.
- [KUDO] When El Pais has a stagnant economy
and an impoverished middle class
what this tax reform brings.
[Gustavo speaking in Spanish]
- [KUDO] It is to increase taxes
on the productive company,
small and medium fundamentally.
[Gustavo speaking in Spanish]
- [KUDO] An absolute contradiction and absurdity.
- The artificial intelligence creates these little islands,
these little bubbles that read like complete thoughts
but are not because there's a flow.
There's a flow that's coming after the pause.
And thanks to an absolutely aggressive social policy,
making a gift to the bankers.
- [KUDO] And thanks to an absolutely
aggressive social policy,
how to issue money to give to bankers.
- So the pauses in this particular speech seem
to throw the artificial intelligence off.
You have the vocal inflection.
Increasing taxes on those who govern Colombia.
That you know allows you to understand
there's something else coming.
I'm not finished yet.
Unproductive capital, those who don't work
but prey on the state, on the people and on nature.
Thank you, sir.
- [Narrator] Test three.
We gave Barry and KUDO a 2021 speech
from El Salvadorian President Nayib Bukele.
The speech is highly crafted, prepared
and read from a teleprompter at a high rate of speed.
- We are going to make use of BANDESAL.
I will tell you that it was a very challenging experience
for me as a professional interpreter
to interpret that speech accurately and completely.
What you have to do is process that information
and extract the ideas to be able
to give your listener something that's usable
for them to be able to follow from that huge amount
or that torrent of words that's coming at you.
- [Narrator] Now let's evaluate the speech translator.
- Salvadoran people, good morning.
As you all know,
the world has suffered an increase in prices.
- When I was able to go back and listen
to the AI interpretation, what I noticed
throughout the entire speech
is that it did not miss any of the content.
- [KUDO] The cost of these goods increases
to match the supply of new dollars,
new euros, new yen, and therefore,
that affects world prices.
[Nayib speaking in Spanish]
- [KUDO] That is why we are experiencing a global inflation
that hits the pockets of everything, of all the people
in the world.
And El Salvador is no exception.
- I was taken aback
by the artificial intelligence's ability
to get all of the content and hold onto it.
So in terms of completeness, it was very good and better
than what I was able to do at that rate of speed.
However, it often had mistakes and challenges
with making the syntax, the word choice,
and sometimes the grammar in English,
and sometimes it would choose words
that weren't really the most accurate way,
the way a native English speaker would say something.
It talked about new currency issues.
- [KUDO] This inflation generated by the incorporation
of new currency issues, usually dollars, euros, yen.
- Which was really ambiguous
because it was talking about new infusions
of cash or currency into the monetary system.
And people could confuse issues
with problems and think, okay,
what's the problem with these new currencies?
What is happening is that the process
that the professional interpreter has been doing
for as long as professional interpreters have been working
is to process that information
and then serve it up to the listener already processed
to know this is what's being talked about.
So rather than saying new currency issues,
I would've said infusions or injections
of more cash into the monetary system.
- [Narrator] So for a profession that started
with the Nuremberg trials and continues to be used
for important meetings and negotiations today,
when is it appropriate to use an AI solution?
- I would say in informal situations, situations
where there is not a high level of consequence for mistakes.
I would not trust this in courtrooms.
I would not trust this in the legal system.
- If there is a decision that has to be made,
a medical interaction, asylum cases.
- I don't know that I would trust it
for situations that require metaphorical language,
that don't catch the figurative meaning.
- Do I want to have a high-level diplomat
or a president of my country speaking
to another president through AI to make important decisions
about the future of literally millions of lives?
No. - We need to look at it.
We need to examine it.
We need to find out how we can use this constructively
and how we can use it carefully to aid us and not be afraid
of it, because that's no use if we're not flexible.
We're eminently in a world where we need to be flexible
and we need to be open
to new ideas and new ways of doing things.
- It is not a replacement, and you have to understand that.
And if we're able to understand the power of the tool,
we can respect the tool and use it to everyone's benefit.