So let’s talk about robots. Robots can be really good at a lot of things, like flying,
or fighting, or making you think deep and discomforting thoughts about what it means
to be human. When it comes to natural language, though, the track record of our artificial
friends is not quite as good. Still, it’s pretty amazing how far we’ve come in such
a short time. I’m Moti Lieberman, and this is the Ling Space.
When you start looking into the stuff that happens around us every day, you might think
we’re living in a crazy science-fiction future. We’re getting high-definition photos
all the way from Pluto. We can control artificial limbs using brain implants and the power of
our thoughts. And, we can talk into a little device many of us carry around, and pull up
a staggering amount of information about the world.
But how does talking at something like Siri work? I mean, your smartphone doesn’t actually
understand you. What’s programmed into speech recognition software that makes it possible
to turn wordy mouth-noise into a reliable string of characters on a screen? The answer
is, at least in part: linguistics.
Linguists know all this stuff about how sounds are made, right? We can break down a vowel
like /i/ into its component waveforms, recreate it using technology, even identify it at a
glance once we’ve done some training. So it should be a breeze to design software that
can pick out those sounds from your speech stream and transmogrify them into written words.
Of course, if you’ve ever tried out a smartphone’s voice-to-text function, you probably noticed
that the transmogrification isn’t … flawless. And you need only look at YouTube’s automatic
captions for a while to get some real gems of speech recognition nonsense.
That’s why it’s so important to us here at the Ling Space to make sure we caption
all of our videos ourselves.
Making fun of speech recognition errors can be pretty entertaining, but when you think
about all the variables involved, like variation between speakers or even between different
versions of the same sound, it’s super impressive just how close our tech actually gets.
Early speech recognition was getting its legs back in the 1950s, using exactly what we mentioned
earlier: the particular profile of the soundwaves in each vowel. The early stuff was all speaker-dependent,
which means you had to train it to recognize the waveforms of one specific speaker, if
you wanted it to spell out what you were saying with any accuracy at all.
And even then, all the early programs could recognize was the digits zero to nine in English.
Which doesn’t sound like much, but given that top-of-the-line machines at that time
had about 10 KB of RAM and filled up a huge room, that’s actually quite a feat.
Speech recognition started to pick up steam in the 70s with some fancy math that helped
computers make the most of patterns in human speech. These methods and the ones after them
used a sophisticated kind of statistical modeling that drew from real-world knowledge about
how sounds pattern together, like how likely one sound is to transition into the next.
Like, in English, combining [k] and [t] works way better within a word if [k] comes first,
like doctor [dɑktɔɹ] or wrecked [ɹɛkt]. If you combine [tk], that’s almost certainly
a boundary between two words, like robot call [ɹowbɑt kɑɫ], or get crazy [gɛt kɹejzi].
But it’s not just knowing the phonemes and where they can show up that's useful. You
can get a lot of really rich information to identify words by looking at the effect that
a sound has on the ones around it. Like, the [ɑ] sound in ‘walk’ sounds a bit different
than the one in ‘tall’. That’s because a little tiny bit of lip rounding from that
[w] carries over. When you include stuff like that into your statistical model, your
speech recognition gets harder and harder to piece together accurately, but more and more precise.
Once each phoneme or word is associated with a statistical distribution of how it can sound,
you end up with a thick web of probabilities, that gets thinned out as you move along the
speech stream. Lots and lots of words start with /æ/, for
example, but once you register all the phonetic properties of the next sound in a word like
‘Adam’, your options shrink down to just a few.
Knowing phonological influences is just part of the picture, though. Ideally, you want
your computer to be able to recognize more than just individual words. You want to be
able to present it with a continuous speech stream and have it pick apart the words that you’re
saying. That’s where programming it with knowledge of syntax does some real work.
Understanding how words fit together in natural language is super useful for resolving ambiguous
streams of sound. Like, “get in the robot” or “get in there owe bought” might sound
the same, but only one of those can be a sentence of English.
Speech recognition programs made it out of the labs and into the homes of consumers by
the 1990s, but the way the algorithms got set up still was hugely time-consuming and
labor-intensive. And, in spite of all the hard work of the
researchers, there still was a lot of room for error. Because each rule had to be put
in by hand, it was really easy to miss things, or code in some rules that had the wrong results.
Getting punctuation right was especially hard: you don’t want your computer transcribing
something like “I got the message you left me” into “I got the message. You left me”.
So people got to thinking about how to improve the whole field. And a few years ago, they
found a technique to make speech recognition better. And we mean like, *way* better. The
basic idea is, instead of spending tons of time teaching a computer all the different
sounds of your language, and the hundreds and thousands of little rules that determine
how they go together, why not just make a computer that can learn, and then let it teach
itself all that stuff?
This is called “deep learning”, and it actually aims to reproduce what’s going
on in a human brain a lot more closely than the statistical models that we talked about earlier.
And to get to that, we’ll have to talk about the brain a little bit. Which is great, because
I love brains!
Brains are amazing, because each neuron, each individual cell, is interconnected with a
bunch of others, and although each one responds to just a tiny bit of information, the way
it interacts with the others in parallel makes for a huge amount of processing power. And
these neural pathways change or adapt to feedback, strengthening or weakening connections
depending on what response they get for their stimuli. What you end up with is super fast,
super efficient processing of information. It's part of why we can communicate the way that we do!
So scientists have managed to make computer programs that work on the same concept as
our awesome brains. They even call these “neural networks”, in honour of our own cognitive
machinery! In artificial neural networks, each “neuron” is represented by a mathematical
function that takes a bit of information, reacts more or less strongly based on its
prior experience, and then sends a signal along to the other artificial neurons within
its sphere of influence.
Take our ‘walk’ and ‘tall’ difference from earlier. Instead of needing an actual
human to teach the program to look out for this, and every other tiny difference between
speech sounds in different environments, a deep learning model will listen and learn,
figuring out the pattern on its own based on lots and lots of data.
The data is provided by humans, but which sound differences matter in telling words
apart is something your artificial neural net can determine for itself. By working out
a ton of possible connections in parallel, and then strengthening the ones that give
you meaningful results, you end up with a system that learns and adapts through experience, just like we do!
Let’s compare it to something we already talked about in a previous episode: priming.
When you encounter a sentence, the rules of your language as well as all your experience
with other sentences help you predict what the next word will be. It’s the only way
to deal with the speed of information coming your way, and although it’s not always completely
foolproof, it does make us humans really good at recognizing speech. Every word you’re
hearing or reading from me right now is triggering a whole explosive network of electricity inside
your brain, as your refined neural circuitry uses what it already knows to speed up comprehension.
What deep learning does is allow software to process sentences in a similar way. You’ve
got an initial setup of rules, which maybe you could compare to the original parameters
of Universal Grammar that every human baby is born with. But after that, how it applies those
rules to parsing out incoming speech is an acquired skill. Everything it hears becomes
a probability tool, and it builds something similar to a grammar inside its own virtual head.
Of course, this still involves a lot of work from the human researchers. Because it’s
always learning to tell things apart more carefully, you need people there to make sure
your program is on the right track. But deep learning works really well, and what you end
up with is something that’s way more precise and makes way fewer mistakes. Over the past
few years most of the really big speech recognition things you might know, like Siri on your iPhone,
Google, or Microsoft Cortana - have been using the deep learning model. And, for example,
just this past summer, Google announced that since switching to deep learning, their error
rates got cut by half. 50%! That’s a huge improvement.
We’ve come a really long way from a machine the size of a small apartment that could understand
the numbers one to ten. Everyone with a smartphone has access, in the literal palm of their hands,
to a really sophisticated artificially intelligent speech recognition system, that will learn
and adapt to the data you put in. Siri can help you out in 17 different languages, and
Google voice search understands almost 40. Every time you call your credit card company
or airline and end up navigating through a menu to the tune of ‘Yes’ ‘No’ ‘Billing
inquiry’ and the like, you’re actually interacting with an AI that’s the fruit
of decades of fascinating research.
So when you’re dealing with your phone, think about how good it’s gotten and how
far it still has to go. And remember how much work’s gone into making your computer able
to interact with you just using your mouth sounds. They may not be perfect yet, but you
have to respect the robots.
So, we’ve reached the end of The Ling Space for this week. If you correctly processed
my words, you learned that speech recognition software recognizes sounds based on their
acoustic properties; that statistical modeling pays attention to the likelihood that one
sound or one word leads into the next; and that more advanced programs use deep learning
models that try to emulate the way our own neural systems work.
The Ling Space is produced by me, Moti Lieberman. It’s directed by Adèle-Elise Prévost,
and it’s written by both of us. Our editor is Georges Coulombe, our production assistant
is Stephan Hurtubise, our music is by Shane Turner, and our graphics team is atelierMUSE.
We’re down in the comments below, or you can bring the discussion back over to our
website, where we’ll have some extra material on this topic.
Check us out on Tumblr, Twitter, and Facebook, and drop by our new store! And if you want
to keep expanding your own personal Ling Space, please subscribe. And we’ll see you next
Wednesday. Agur!