Hi! Welcome back. If you are here after viewing the previous video where we talked about the
different branches of AI, or if you’re landing on this page for the first time and are wanting
to learn about NLP, then you are in the right place. NLP, as you may know, stands for Natural
Language Processing. In this video, we’ll discuss what NLP is, how it works, some examples,
and a few applications. The video is about 10 minutes in length, but stick around, there
are interesting details you’ll learn. If you wish to jump to specific topics, the description
below captures the timestamps. So we all know that humans have always had the ability to
communicate with others through a common language. And we have always been able to breeze past
differences or errors in speech such as mispronunciations, accent changes, different contexts, homophones,
slangs, slurs, and much much more. All differences aside, we are still able to understand each
other perfectly. This is what makes humans and human language so powerful. But what about
computers? Well, you might argue that machines have always had the capability of understanding
language, as in their own machine language (known as code). But since humans have to
manually write all this code, it makes the language extremely limited in vocabulary and
severely restricted. For example, if a programmer makes a small spelling or syntax error, the
program will crash and the problem won’t be solved. To better understand the big issue
with these limitations, I want you to picture this. Nick here is a vet and over the past
few weeks, he has observed that many of his patients have been getting a new illness.
To warn other pet-owners about this disease and prevent other pets from contracting it,
he wants to figure out exactly why and under what conditions animals get the disease by
analyzing his previous cases as well as cases on the internet. So, he starts to look at
his data - pictures, notes, sketches, essays, voice recordings, and more. Suddenly, he feels
overwhelmed because going through and making sense out of all of this will take ages and
the worst part is, your computer or any other technology can’t exactly help you. You see,
Nick knows that in the documents, foot is referring to the part of the body and not
the unit of measurement that and date means a specific point in time rather than the fruit,
but a computer doesn’t know how to decide this because it can’t understand the context
that words are in. Afterall, understanding a word all boils down to understanding the
words around it, because a word by itslef doesn’t have much significance – it all
depends on the context it’s in. Because of this, the computer can’t make sense out
of the data, and ultimately can’t really help you. But given how powerful computers
are in terms of memory, speed, and intelligence, imagine just how much more useful they could
become if there was some way for them to understand and process our language instead of only incomprehensible
1s and 0s. Well, no need to dream about this anymore because with Natural Language Processing,
this is the new reality. With NLP, new doors open. Nick can now analyze the heaps of unstructured
data within minutes. Smart assistants, like Siri and Alexa, can understand our words and
context clues to improve our lives. Google can not only predict results as we start typing
in the search bar, but can also look at the whole picture and recognize more than just
the words we type. Our phones and computers can predict what we will type, finish our
sentences and suggest relevant words. NLP can solve bigger problems as well, such as
crimes and even diseases, by identifying clues and patterns in emails and written reports.
So as you can see, NLP is extremely powerful and if harnessed correctly, can improve our
lives many-fold. Now that you know what Natural Language Processing is, how does it actually
work? Well, let’s dive in. Every day, we each say hundreds of sentences and we talk
to many people. In school, we all learned about the parts of a sentence and which types
of words come together to create a sentence that makes sense. We know that a sentence
is ultimately made up of a noun phrase and a verb phrase, but we can break the noun and
verb phrases up into individual parts of speech. For example, in the sentence “The man leaves
the bank”, THE is an article, MAN is a noun, LEAVES is a verb, THE is an article, and BANK
is a noun. From here, we can combine THE and MAN into a noun phrase and THE and BANK into
another noun phrase. The verb LEAVES and this noun phrase can turn into a verb phrase, and
finally, this noun phrase and this verb phrase become a full sentence. The techniques used
to create this parse tree broadly fall into the categories of part-of-speech tagging and
chunking. Now, you might be wondering why I’m taking you back to elementary school,
but it’s because these parts of speech are what give Natural Language Processing the
power of understanding context. While an ancient computer might have been confused as to whether
the word LEAVES is referring to the act of exiting a place or the plural of a leaf, and
whether the word BANK is referring to a building that stores money or the land next to a river,
a computer powered by NLP will see the fact that LEAVES in this context has to be a verb
and not a noun. By analyzing hundreds of sentences and different word patterns, it can figure
out that it is far more uncommon for a person to say that they are leaving a bank of a river
than it is to say leaving a bank that stores money, so it must be this bank. A computer
can see that the main subject of this sentence is the man and he is doing the action of leaving
a place, which in this case is a bank. And it’s not just part-of-speech tagging and
chunking that allow NLP to figure out the meaning of a person’s words. There are also
other techniques, some briefly mentioned in one of the intro videos, that can broadly
be classified into two categories: syntax and semantics. More on these in the next couple
of minutes. It’s these individual words, their parts of speech, and their placement
in the sentence that give the computer knowledge and context as to what the sentence is really
trying to say. You might be thinking, well this makes sense, but how does the computer
actually know the parts of speech and how can it figure all this out even if it knows
the parts of speech? Well, this is where the power of today’s society comes in, specifically,
the power of the big data available in today’s world.Basically, a programmer can gather up
all sorts of words, phrases, sentences, grammatical rules, and word structures, and input all
this data into a Machine Learning algorithm, which we’ll talk about in a future video.
Using all this information, this algorithm can learn what words usually end up next to
each other, how a sentence should be formed, why certain words fit into a sentence better
than others, etc etc. And this, is precisely how a computer will eventually figure out
why leaves makes more sense than leaves in that sentence. And data is collected for NLP
algorithms every day. Ever noticed how, when you call certain companies, a robot says “this
call may be recorded for training purposes”? Well the conversation on the call traditionally
serves as legal data and data that helps companies improve themselves. But in the future, this
data can also be fed into NLP algorithms to make them even more intelligent and effective.
And this is not just a futuristic possibility. Some companies are already implementing this.
And just like this, there are hundreds of ways that you, even without knowing, are contributing
to make this technology even more powerful to improve society. Now back to what I mentioned
a couple of minutes ago: Syntax and Semantics, the broad categories under which NLP’s techniques
fall. Let’s take a look at these now. So tokenization can be split into two categories.
Sentence tokenization and word tokenization. Sentence tokenization is separating a paragraph
into distinct sentences and word tokenization is separating a sentence into distinct words.
This allows the computer to learn the potential meanings and purpose of each unique word.
Next, we have stemming. Stemming is the process of reducing a word to its root, or stem. It
does this by chopping off universal prefixes and suffixes such as es, s, ing, and ed. Stemming
is a powerful technique, but this crude chopping solely based on common prefixes and suffixes
sometimes cuts necessary components of a root and changes the meaning of the original word.
Well, this takes us to our next technique, which helps solve this problem. Instead of
chopping off beginnings and endings, Lemmatization reduces a word to its root form by analyzing
the word morphologically. Let’s take a look at what this means. So if I have the words
am, are, and is, the root form for these words is be, which can be observed through lemmatization.
Stemming, on the other hand, would not have been able to figure this out, as chopping
any letters off of these words would not have outputted “be”. Now for semantics. So
Named Entity Recognition allows the machine to categorize specific words in a sentence.
This is especially helpful for words such as Google, which can be referred to as an
organization, or a verb. And finally, for Natural Language Generation. NLG is the process
through which a machine produces natural human language. It uses math formulas and numerical
info to extract patterns and data from any given database and output understandable human
language text. And to summarize, Natural Language Processing enables a machine to take what
you are saying, make sense out of it, and formulate a statement of its own to respond
back to you. As we have seen, it uses a variety of techniques to accomplish this: part-of-speech
tagging, tokenization, stemming, lemmatization, named entity recognition, and natural language
generation. As a result, NLP allows computers to understand context and meaning of our words.
So now you know what Natural Language Processing is and how it works. But there’s still one
question left. how is the computer able to even understand your sounds in the first place,
or in other words, recognize your speech? Well, this is exactly what we’ll be talking
about in the next video, so stayed tuned for that. Make sure to give this video a thumbs
up if you’ve learned something and comment down below what other words could confuse
a non-NLP machine, such as leaves vs leaves. And if you are new here, then make sure to
subscribe and turn on notifications so you don’t miss any videos that’ll help you
on your journey towards mastering artificial intelligence. Thanks for watching, and I’ll
see you in the next one.