SARAH WELDON: Hello, and welcome to Engage and Translate--
Text and Audio Chat in Over 100 Languages.
I'm Sarah Weldon, Product Manager for Google Cloud
Translation Services, which include
APIs for pre-trained and customizable machine
translation models, and a new beta API for supporting
streaming audio translation.
More on that later.
I'll be joined later in this talk
with Dale Markowitz, Applied AI engineer at Google Cloud.
And she'll bring some of the features
that we are discussing to life in a couple of demos.
Today's companies, their employees, and their customers
are truly global.
And every day, they're also more digital.
Keeping pace with the exchange of information
in multiple languages at scale is a growing challenge,
especially so when more work is moving to virtual.
So companies are turning to machine learning
to augment their workloads and to make content more accessible
faster than would be possible without automation.
Luckily, advances in machine translation
continue to drive up quality.
And new features on our products give customers
more control over how their content is translated.
More on that later.
Let's take a quick look at where we started.
Pre-2016, the world is relying on statistical machine
translation.
As an example for this, we take a set paragraph
from a Japanese language and we translated it into English
for readability.
So in this case, this translates as, Kilimanjaro
is 19,710 feet of the mountain covered with snow
and it said to be a highest mountain in Africa.
It's a little awkward in how it's written,
as well as there's some words missing
if you are a fluent reader.
But if we fast forward to today, we've done a lot of work
to improve machine translation using machine learning.
And so Google introduced a neural machine translation
after 2016 and into 2017.
And under that new model, this same translation
would come through as, Kilimanjaro
is a mountain of 19,710 feet covered with snow
and is said to be the highest mountain in Africa.
It's a much more fluent translation.
It doesn't feel awkward.
And it's a good number of words.
But we keep improving, even beyond this.
In summary, the last major jump in quality we saw
was in 2016 to 2017.
But we've continuously improved over the years as well.
And in the most recent year, which you see with this graph
in green, we saw an increased jump over multiple languages
through a number of different technology investments
driven by our research teams, and an average
of 5-point blue score jump over 100-plus languages.
And this improvement is comparable to the gain
observed years ago when translating
from statistical to neural machine translation, as I
spoke about a few minutes ago.
So our products include Translation API Basic,
Translation API Advanced, and AutoML translation,
and then the new one that I was talking
about a little bit later.
For Translation API Basic, the implementations on it
are really intended for simple use cases.
It's intended for user-generated content
where you're not trying to control
a lot of the specific translations.
So we wanted to make it very simple to use.
And so Translation API Basic does do important things still
with language detection.
And it also continues to use the advancements of the new machine
learning models as we release them and develop new languages.
But you can use Translation API Basic
for chat scenarios and user-generated content,
because it's easy to integrate with
and you're not trying to change the translations after the fact
for a number of these cases.
But in this user-generated content,
it still is really easy to use and will
enable you to easily integrate with many languages
at a very simple platform.
As an example case, I'll tell you
a little bit about a gaming scenario, where
we had one of our customers, in this case, ELEX,
which publishes games in mobile gaming chat.
And their gamers are international.
And so when the gamers are wanting to talk to each other
and collaborate in order to achieve certain goals,
or to just chat online, they sometimes
need to be able to cross between the multiple languages.
And so by integrating with the Translation API Basic,
ELEX is able to improve the engagement experience
for their end users and be able to improve the overall gaming
experience just by connecting in with our APIs.
But this isn't just specific to gaming or to tech.
It's actually really useful for government scenarios as well.
So let's talk about another customer scenario,
this time, public sector.
Within the US, many languages are spoken,
and governments are required to make
information accessible to their constituents.
Let's look at how machine translation
can help with this scenario.
So I'll turn it over to Dale.
DALE MARKOWITZ: Thanks, Sarah.
I'm Dale.
I'm an applied AI engineer here at Google.
And I'm talking to you from Austin, Texas.
Texas is a really interesting state
because it's actually the second most multilingual state
in the US, after California.
And something like one in three people
don't actually speak English as their first language
in their homes.
So it's really important, especially in
times like these, that the city is
able to communicate with constituents
in their own languages.
So I recently had heard about this project.
It was actually done by the state of Hawaii,
where they used the Google Translate API to automatically
translate a bunch of legislative documents
so that all of their constituents
could understand the documents.
So I kind of wondered, could I do the same thing for Austin?
So I decided I would go to the Austin website,
and I found a bunch of different questions that people ask.
Where do I take out my garbage or how do I
report a leaking fire hydrant?
And I decided to build a chat bot that
understands questions and then gives the city's answers.
But I used the Translation API to be
able to detect what language people are asking in,
and then respond in their own language.
So let's take a look.
This is a chat app that I built to answer questions
about the city of Austin.
It recognizes lots of languages.
So for example, if I ask how to check
if I'm registered to vote in Spanish,
it will detect my language and use the Translation API
to respond in the same language.
The Translation API supports over 100 different languages
and lots of different character sets,
so you can write your app code once and have
it work all over the world.
So the demo that I just showed you,
it was working with a chat bot that had some preset answers.
But actually, the architecture works exactly the same
for any sort of chat--
like for example, if I was building a chat
app to chat with Sarah and we wanted
to use two different languages.
So let me walk you through how it works.
So the first thing you might have noticed
is that you didn't actually have to specify what language you
wanted to use chat in.
That's because the Translation API,
if you give it enough text, can actually
tell what the language is.
So the text gets sent from the chat client to a server.
The Translation API detects the language.
And then it translates the sentence to English
because my chat bot only understands English.
Then once the chat bot replies, we
use the Translation API again to translate back
to the sender's language and send the response back
to the chat.
And that's how you can build a translation chat app.
Now, this worked really well for the majority of the cases
that I tested.
But there were some where it didn't do that well,
in terms of translation quality.
And those are the cases where there was some really specific
Austin terminology, like the name of our metro system
or some really specific words for things in Austin.
So I'm going to hand it back to Sarah,
and she'll tell you how you can improve quality
for domain-specific cases.
SARAH WELDON: Thanks, Dale.
That was great.
Now let's talk about what we can do
to modify content to better represent
different possible translations within a language,
depending on the industry context or location.
So in this scenario, I'm thinking of public sector
again.
And it's a pretty common thing to want
to be able to provide comments to a public agency--
in this case, let's say the attorney general.
Now if you were talking to users that
are in Spanish, that are speaking Spanish,
there is a pretty common translation
for attorney general, which is fiscal general.
So in this case, our machine translation
would automatically select this for the translated response.
But there are other countries, say in Latin America,
that might have a different version of what they call
their attorney general's office-- in this case,
Ministerio Publico Fiscal.
But even in California, they actually
have a pre-described glossary.
And if you look at that, they refer
to their attorney general's office in Spanish
as Procurador General.
So what if you could customize your translations
to your domain?
With Translation API Advanced, you can actually do this.
And if you have a terminology base, which
a lot of public organizations do, and companies do,
as well, then you can use this as your starting point
with an easy integration with Translation API Advanced.
Because you have your English terms and your terms
in other target languages, if you just
create this into a CSV file and import it into your Google
Cloud project, then you can use this
with future cases for machine translation
just by referencing your glossary in your Translation
API call, and then when you get the response back,
it'll actually preserve your preferred term
over what the machine translation might
have otherwise predicted.
And Translation API Advanced also
has other features that can help with both scaling
your implementation, as well as more advanced customization
capabilities with our custom machine translation models.
All of that can be done through one API integrated
with your GCP project.
But let's say you have even more than just terminology
and you want to go further.
You're company or organization that
has a lot of past human translations.
And you want to be able to use those translations to improve
the machine translation prediction for what
a translation would likely be for your content, your context,
or your industry.
So in that case, it's really important
to understand conceptually that Google
has both pre-trained models, which is what you typically
use with machine Translation API Advanced or Translation API
Basic.
And these are pre-trained models that we've created
using a lot of training data.
But if you have your own training data,
you can actually use those as a starting point,
and then build and improve on them
with a custom training on AutoML Translation.
So by taking your past human translations,
bringing that to our platform, and running trainings on it,
then you're improving the statistical and mathematical
likelihood that our model is going to provide back
to you a translation that's tuned to your context
and to your industry.
So it does this through transfer learning.
And it has an entire platform where
you can go and learn and manage more of these models
to integrate it with more advancements in your platform.
But now, let's go back to our chat scenario.
Our chat app uses the right language, but it doesn't speak.
It's actually just using text translation.
So since users both write and talk,
there are scenarios where we might want to have audio,
as well as text translation supported for our use cases.
So can we translate speech in real time?
Organizations are seeing value in delivering their products
and content globally.
And it's not just in text.
It's in video.
It's in audio.
It's in calls.
And so for these scenarios, you want
to be able to have an easier way to implement translations
for those use cases.
And it's applicable in media entertainment.
It's applicable in technology.
And it's applicable for service providers
that might be trying to support customer support cases.
But what's so complicated about translating nontext data
with audio and video?
We have our speech APIs and our video APIs
that allow for interacting with those types of content.
And we have translations, so we can just
integrate between all three of them
in order to be able to get an outcome.
But it's kind of hard to do.
And it comes with other challenges
that are nearly impossible to solve
on the developer-client side.
So we're looking at how to solve that on the cloud side.
For one thing, latency can be a lot higher
if you're integrating across multiple APIs, especially
if one is set up for streaming and the other one
is setup for more synchronous cases, where
it's waiting for the content to finish
before it starts to figuring out what a translation would be.
Additionally, you have to do multiple API calls.
So that can be a complexity for developers
to work with, especially if they're trying to figure out
how to improve accuracy, et cetera,
across these two different APIs.
And then you're also dealing with a streaming client,
which creates some complexities with integrations
and usability.
And it's a challenge in itself.
So Media Translation API simplifies this process.
It supports directly audio files or microphone input,
say through a mobile phone or through your browser.
And it also accepts that content directly,
so you're not having to run through a speech model first
and then a translation model in order
to get a translation back.
You can go directly to Media Translation
API with your audio, and you'll get back
a text translation in the language which
you've set as your target language for translation
outcome.
In addition to abstracting away the frictions of multiple API
calls, we have some other features that
overall improve the experience.
So automatic punctuation detection,
as well as using enhanced models for video
or telephoning use cases allows you
to be able to make the most to get the best quality optimized
for your audio translation outputs.
And then we also have language support,
because it's a translation model.
So we support French, German, Italian, Hindi, Portuguese,
Russian, Spanish, English, and others,
for you to be able to translate multiple languages
for your user base.
So with that, I'll take it back to Dale
so we can see the Media Translation API in action.
DALE MARKOWITZ: Thank you, Sarah.
So as a developer, I can confirm that it
is tricky to do anything in real time, especially
real-time translation.
But the Media Translation API makes this a lot easier,
which is great, because there are so many times I want
to do real-time translation.
Like imagine if I'm doing a live stream
and I want users all around the world to understand it.
Or maybe I'm giving a talk in person
and I want different audience members
with different languages to get what I'm saying.
So I'm going to show you something
that I built with the Media Translation API
and the Raspberry Pi and the Mini Projector.
And let's take a look.
Hi, my name is Dale.
I'm using the Media Translation API and a Raspberry Pi
to translate what I'm saying in real time.
So that is what I'm bringing to my talk next time
I give a talk somewhere that speaks a different language.
And with that, I'll give it back to Sarah.
SARAH WELDON: Thanks, Dale.
I wanted to close with some thoughts from one
of our translation customers for Media Translation API.
When they integrated with our Media Translation API,
they were able to see games in low latency,
as well as the overall quality of performance.
And so they're able to provide real-time streaming
translation for video chat.
OnePlus is a phone provider.
And they've created a beta application
that allows for their customers to be able to opt in and be
able to see the translations from the person
that they're speaking in a different language
while they're still engaged with them in video chat.
And so with this, we're able to demonstrate
how text translation and audio translation
can work seamlessly together to create a better end
user experience.
To close, I wanted to point you to our documentation
on our Cloud website, where you can get more information
on AutoML translation, Translation API
Advanced, and our new Media Translation API.
Please check them out.
Thanks for joining us.