Hi, I'm Emily Chiu, a Creative Producer at Duolingo.
Later, I'll be joined by Kevin Lenzo, Research Scientist on the Speech Lab.
Last year at Duocon, we introduced you to our characters and their names.
This year, we'll introduce you to their voices.
You're probably familiar with Siri or Alexa's voices,
these super-iconic recognizable computer-generated voices
that you hear everywhere now.
These are produced by having a real person, usually an actor,
read thousands and thousands of lines.
The result is a computer-generated voice
that can take any text you give it and "read it" in that person's voice.
In a nutshell, this is what Text to Speech, or TTS, is.
Duolingo also uses TTS.
We don't record every single line we have in the app.
For a long time in our app, we used generic or stock TTS voices
that you could purchase just like stock photography, for example.
So, here's an example of how our character Junior, who is an 8-year-old boy,
sounded like in the Spanish to English course.
[adult male voice] Drugstores sell both shampoo and conditioner.
As you can tell, that doesn't really sound like an 8-year-old boy.
Now, we're working on giving our characters their own distinct voices
that reflect their individual personalities.
Here's an example of what Junior sounds like now.
[child's voice] Drugstores sell both shampoo and conditioner.
Junior is just one of our ten Duolingo characters.
We've been developing them for a year and a half.
And we've always had Duo, but now we have nine other human characters
that accompany you from Stories to lessons.
[Emily] Two of my favorite characters are Lily, the thoroughly unimpressed,
surly-looking teenage girl in the center with the purple hair
and also Oscar, the mustachioed fancy-pants art teacher in the pink shirt.
I won't go through all the characters now,
but they all have distinct personalities and back stories we've developed.
One thing we considered a lot when developing these characters
is how to make them iconic.
Super-memorable, distinctive, with really strong brand recognition,
but also really relatable, engaging and fun.
To this aim, we consulted experts
who have created some really iconic characters.
We talked to the studios behind Sesame Street and Adventure Time.
One thing we learned is that characters are iconic because of how they look,
how they move and how they sound.
The art team had done an incredible job with the look and move portions.
The characters were in the app cheering you on, celebrating your wins.
The last piece of the puzzle was how they sound
and the first step to figuring that out was casting.
We decided to borrow some best practices from animation in Hollywood.
We hired a casting director.
She worked on Robot Chicken and Call of Duty,
and we reviewed auditions from established voice-over actors.
So, what were we looking for in these auditions?
The first thing was personality.
We wanted lots and lots of personality.
Because, in our app, most of the content is single sentences and lessons,
we really needed these actors to convey these characters' personalities
really clearly in just a single sentence.
The second thing we looked for was humor and entertainment.
We really just wanted funny, engaging voices
that could make our users laugh and smile.
The last thing we were looking for is distinctiveness.
You'd want to be able to distinguish the characters from one another.
But one thing that's really important when learning a language
is to hear all different kinds of voices.
So adults of different ages and genders
as well as children, who are often the hardest to understand,
especially in a language that's not your native tongue.
So to do this,
we spent many months reviewing hundreds of auditions.
really debating the merits of each.
Sometimes, it got a little contentious.
We were each fighting for the version of Oscar we really wanted.
But after about a year,
we had cast nine characters in English
and we'd even finished recording all the necessary lines for the TTS.
This was a huge milestone for us.
But what about the other 39 languages that we teach?
We decided to move forward in four of our most popular languages.
Spanish, French, German and Japanese.
We very quickly ran into a huge logistic problem.
How do you cast in a language that's not your native tongue?
Most of our team is native English speakers.
We're learners of these languages, but we're not proficient, necessarily.
We're not advanced, fluent or native, definitely.
So we're missing some nuance of the performances.
So we wondered, "How do you capture the nuance
of little boy Junior, on the left, his precociousness in German?
Or emo teen Lily, in the center, her sarcasm in Japanese?"
We took a really analytical approach to this issue.
We assembled a team of native speakers,
some creatives and some language experts,
and we had them evaluate specific qualities in the auditions.
For a Lily audition, for example, we would ask them,
"On a scale of 1 to 5, how disdainful does this actor sound?"
Or for Oscar, the fancy-pants art teacher, we would ask them,
"How pretentious does this character sound?"
And we ended up with actors in all the other languages that sounded
pretty similar, in some cases, to the English actor,
and, in some cases, they were like the French version of that character.
Right now, I'll play for you the English version of Oscar.
[pretentious] My father and my mother like my jacket.
And also the French version of Oscar.
[pretentious] Mon père et ma mère aiment mon bluson.
So, we are almost finished recording these
and we're actually building the voices now.
So I'm gonna pass this off to Kevin, to talk about building the voices.
Thanks, Emily.
All right. Let's talk about building synthetic voices.
We use machine learning to make these voices talk.
[Kevin] That means we analyze all of the text that goes into it,
and we predict how a voice should say everything:
the phrasing, the timing, the intonation. Everything.
Once we do build it out of these amazing materials Emily just talked about,
with characters that bring everything to life,
then we make sure it's up to our quality standards.
We thoroughly evaluate everything.
If there's something that doesn't work,
we try to improve the algorithms
or correct, remove and edit the data that goes into it
to make it just right.
One example might be
if a question doesn't rise the way it should at the end of a sentence.
The building blocks that go into this
are those 6,000 aforementioned sentences.
These are carefully designed and completely balanced
for combinations of speech sounds in different contexts
with phrasings and different sentence types.
For instance,
the sounds in speech sound different in different contexts,
like the initial L in the word "lateral" sounds different from the final L,
and the same is true for the initial and final R in the word "reader."
The P in the word "peak"
is pronounced differently from the P in "speak."
We want to get all these things correct,
because we want to teach as if they're learning from a native speaker.
To get all the context that we'd need,
we'd need about 100 times the amount of data we collect
for the 6,000 sentences.
And from these 6,000 sentences,
we're going to make a lot of content in all of these voices.
Once we build the voices, we still have to say everything correctly.
And language can be ambiguous.
For instance, the English word "r-e-a-d" can be pronounced "reed" or "red,"
depending on whether it's the present or the past tense.
Or the phrasing may be wrong.
For instance, we might hear,
"She never saw a dog. And didn't smile."
Which means something very different from, "She never saw a dog and didn't smile!"
And another simple example would be if I were to say your name.
"Hello. Kevin."
As opposed to, "Hello, Kevin."
Every one of these examples is corrected by a linguist.
So when we see or hear something that's wrong in the course content,
we can fix it.
Now, why would we build these voices?
The key is really flexible content.
We can bring new content to life,
we can fix things, we can add new content
and we can respond to what a learner is going through.
We can make new lessons, depending on what they're doing at the time.
It adds depth and brings that content to life, as we've seen before.
In short, they sound great,
they help us be nimble and they open up new learning experiences.
Well, that's enough of us talking!
Now let's hear from our characters, who can speak for themselves.
We each have our own personalities,
and we have back stories and relationships
which help to add depth to our characters.
It makes language learning more fun.
And provides lots of variety for learners to hear.
Not just one or two, but many different people's voices.
Putting everyone together brings language learning alive.
Get to know our stories
as you learn to read, write, listen and speak with Duolingo.
-Gracias. -Merci.
Hasta la vista.
Now that our characters have said thank you for themselves,
we would also like to thank you
and everyone who helps bring these characters to life.
Thank you.