welcome everyone to free code camp i
chrisley on behalf of edureka will take
this session on natural language
processing popularly known as nlp now
edureka is a global e-learning company
that provides online training courses on
the latest trending technologies so
without any further delay let's have a
look at the agenda for this session
so i'll start off by explaining the
evolution of the human language then
we'll understand what is nlp and how it
came into the picture moving forward
we'll have a look at the various
application of nlp in the industry and
next we'll understand the different
components of nlp and the difficulties
faced while implementing those and
finally i'll finish off this video by
explaining you guys the various steps or
the paths involved in the natural
language processing along with the demo
for each one of those steps
so the success of human race is because
of our ability to communicate that is
information sharing using this ability
we have marched ahead of other animals
and have become the most sophisticated
creatures and this is what
differentiates us among all the other
animals we begin to look for ways to
preserve our thoughts feelings messages
and other information we started with
oral communication like other animals
but because of its informal nature we
began painting on walls and caves where
we lived now there was a need to
standardize the drawing so that everyone
could understand and that's where the
concept of developing a language came in
however many such standards came up
resulting in many languages with each
language having its own basic sets of
alphabets combination of alphabets which
were known as words and the combination
of words which were arranged
meaningfully which became the sentence
now each language has a set of rules
based on words and are combined to form
these sentences now these set of rules
are nothing but what we call it as
grammar now i'm not gonna take any more
time explaining you guys about grammar
and all now coming to today's world
according to the industry estimates only
21 of the available data is present in
the structured form data is being
generated as we speak as we tweet as we
send a message on whatsapp instagram
imessaging and various other platforms
majority of this data exist in the
textual form which is highly
unstructured in nature now in order to
produce significant and actionable
insights from this text data it is
important to get acquainted with the
techniques and the principle of natural
language processing so let's understand
what exactly is nlp
now natural language processing that is
nlp refers to the artificial
intelligence method communicating with
an intelligent system using natural
language now it is a part of computer
science and artificial intelligence
which deals with the human language by
utilizing nlp and its components one can
organize the massive chunks of text data
perform numerous automated tasks and
solve a wide range of problems such as
automation summarization machine
translation named entity recognition
relationship extraction sentimental
analysis speech recognition and topic
segmentations now we'll learn about all
of these later in this video now the
goal here is to process i'd rather say
understand the natural language in order
to perform useful tasks some of these
tasks include making appointment buying
things spell checking generating
responses and social media monitoring
now if we look at the various
application of nlp in the industry
firstly we have the spell checking which
is usually there in you can find it
mostly in words or the document reader
even online also you can do the spell
checking now next we have keyword search
and it is also a field where nlp is
heavily used now extracting information
from websites or any particular document
also requires the knowledge of nlp now
one of the coolest application of nlp is
the address in matching which is
basically a recommendation of ads based
on your search what it does is analyzes
the text of the data which you are
already using or searched and match it
with the text data of the advertisement
now sentimental analysis is also a very
major part of nlp another application is
the speaker ignition now here we are
also talking about the voice assistants
like the siri google assistant the
cortana and we need to thank apple for
creating the first voice assistant that
is siri now next we have the
implementation of chatbots now most of
you guys might have used the customer
chat services provided in various apps
now most of these apps use the chatbot
which is it uses nlp to process the data
which we entered and then it provides
response based on an input that is also
an application of natural language
processing now another application of
nlp is the machine translation now the
most common example of it is the google
translate as you know and now it uses
nlp to translate the data or i should
say the text from one language to the
other and that too in real time now nlp
consists of two components
which is the natural language
understanding referred to as nlu and the
natural language generation which is
referred to as nlg now understanding the
natural language involves
mapping the giving input in natural
language into useful representation and
analyzing different aspects of the
language now natural language generation
is the process of producing meaningful
phrases and sentences in the form of
natural language from some internal
representation now it involves text
planning which includes retrieving the
relevant contents from the knowledge
base it involves sentence planning which
includes choosing require words from
meaningful phrases setting tone of the
sentences
and finally we have text realization it
is mapping sentence plan into the
sentence structure
now we'll learn about this later in this
video and usually natural language
understanding which is nlu is much much
harder than energy now you might be
thinking that even a small child can
understand a language so how come it is
so easy for human beings is so difficult
for the computer to process it so let's
understand what are the difficulties a
machine faces while understanding a
language
so in natural language understanding
there are certain ambiguities which are
the lexical ambiguities syntactical
ambiguity and the referential ambiguity
now understanding a new language is very
hard taking our english into
consideration there are a lot of
ambiguity and that to in different
levels now starting with lexical
ambiguity lexical ambiguity is the
presence of two or more possible
meanings within a single word it is also
called semantic ambiguity for example
let's consider the following sentences
and let's focus on the italicized rules
now she is looking for a match now what
do you infer here from the match word is
she looking for a match that is head to
end match or is she looking for a match
as in a partner the fisherman went to a
bank is it the bank where we withdraw
our money or is it the bank where he
rows his boat or he catches the fishes
now syntactical ambiguity in english
grammar this ambiguity is the presence
of two or more possible meanings within
a single sentence or a sequence of words
it is also called structure ambiguity or
grammatical ambiguity now let's have a
look at these sentences the chicken is
ready to eat so is the chicken ready to
eat something or is the chicken ready
for us to eat so this is a kind of
syntactical ambiguity which is often
very hard to info for a new person or
i'd rather say a computer because it
means the meaning of the sentence is
different for the different tones or in
different aspects so for example if i
look at the last statement i saw the man
with the binoculars so do i have a
banner or the man has a binocular
it might be possible that you might be
thinking that i saw the man with
binoculars
means that i have the banners but
somewhere some people might think that
the guy which i am seeing has the
binoculars rather than me so that is
syntactical ambiguity now coming to the
third ambiguity which is the referential
ambiguity now this ambiguity arises when
we refer to something using pronouns now
the boy told his father the theft he was
very upset now when we talk about he was
very upset if you focus on the
italicized word he does this mean that
the boy was upset or the thief was upset
or the father was upset nobody knows
this is referential ambiguity
now coming back to nlp
for using nlp onto our system or doing
any natural language processing we need
to install the nltk library that is the
natural language toolkit so nltk is a
leading platform for building python
programs to work with human language
data it provides easy to use interfaces
to 50 corpora and lexical resources such
as wordnet along with the suit of text
processing libraries for classification
tokenization stemming tagging and much
more
now let me show you how you can download
an ltk
now in order to download nltk just go to
your python shell and just type in the
word nltk dot download using the
parenthesis and then you will get this
type of a window pop-up which is the
nltk downloader you just need to select
all option and click on the download
button and it will download all the
corpora and the
text it has all the packages it has into
one single place and you can choose the
directory where you want to install it
it's better if you download it in your
python directory only it will be easier
for you to access all the files and all
the text which it has to offer now when
we process text there are a few
terminologies which we need to
understand
so the first terminology here we are
talking about is tokenization so it is a
process of breaking up strings into
tokens which in turn are small
structures or units that can be used for
tokenization now it involves three steps
breaking a complex sentence into words
understanding the importance of each of
the words with respect to the sentence
and produce a structural description on
an input sentence so if we take the
sentence today we will understand
tokenization so as you can see we have
five tokens the first one today we will
understand tokenization so all of these
words in computer terms are known as
tokens and this is what is referred to
as tokenization so let me show you guys
how you can implement tokenization using
the nltk library
so here i'm using jupiter notebook
you are free to use any sort of id also
my personal preference is jupyter
notebook so first of all let's import
the os the nltk library which we have
download and the nl ticket corpus
now let's have a look at the
corpora which is being provided by the
nltk
that is the whole data so as you can see
we have so many files
and all of these files have different
functionalities some have textual data
some have different functions associated
with it we have stopwatch as you can see
here your state union names we have
twitter sample data we have different
kind of data and different kind of
functions here
so let's take the brown into
consideration as you can see here we
have brown and brown zip so first we all
we need to do is import the brown and
then let's have a look at the words
which are present in the brown you can
see we have the fluton country grand
jury said and it's going on and on
now let's have a look at the different
gutenberg fields so as you can see under
gutenberg file we have austin mr text we
have the bible text we have the blake
poems the carol alice text
we have the edgeworth parents.txt
shakespeares we have we have the
shakespeare we have the whitman leaves
so nltk is a very big directory and a
very big library to download you might
need a couple of minutes to download
this library
so let's take the shakespeare hamlet.txt
and if you look at the
words which are inside this hamlet file
we see it starts with the tragedy of
hamlet by and it goes on and on so if we
have a look at the first 500 words of
this textual paragraph or what we say
the textual file
so i'm using here for word in hamlet and
i'm using the colon and 500 that is the
end point so as you can see it starts
with the tragedy of hamlet by william
shakespeare 1599 actus primus schona
prima and it goes on and on
so for natural language processing you
can use either of the text which is
provided here for understanding or you
can create your own words
so for example here i have defined a
paragraph based on artificial
intelligence
okay so it goes on like according to the
father of artificial intelligence john
mccartney it is the science of
engineering and so on and on so let us
first define this string okay ai now why
i'm taking a string is because it is
easy to show you guys how to work on a
string so if we do the type ai you can
see it is sdr which is string
now under nltk we have the nltk.tokenize
and we are going to import the word
tokenize
right as a function will understand how
it works
now we will run the word underscore
tokenize function over the paragraph and
assign it a name
let's assign it as ai underscore tokens
so now if we'll have a look at the air
underscore tokens
you can see it has divided the whole ai
paragraph which we gave into tokens
so as you can see it has taken a comma
also into a consideration the hyphen
also and it goes on and on
now let's have a look at the number of
tokens where we have here
so for that we are going to use the
length function so as you can see we
have 273 tokens
now from nltk we have a probability
function which is the frequency distinct
so
i'll show you what it does
so for a word in ea underscore tokens
f-test and we are going to convert it to
lower keys so as to avoid the
probability of considering a word with
uppercase and lowercase as different and
then we are going to assign it a number
this is basically a word count program
and this is being implemented using the
frequency distinct function which is
already present in the nltk library so
let's see what's the output of this one
so as you can see
comma has appeared 30 times
full stop has appeared 9 times question
mark 1 and so on and on like you see
intelligence has appeared 6 times
intelligent sixth time intelligently has
appeared one time
now suppose if you want to know the
frequency of any particular word here so
for that we are going to use the
function f test
that is the distinct frequency so we are
going to use that function over the
particular word so for example let's say
i want to know the frequency of
artificial
so as you can see it is three times
and if we check it in the database you
can see it is artificial it is given as
three
now if you want to have an look at the
number of distinct words here
all we need to do is pass on the f-test
function to the length function so you
can see it's 121. so earlier we had 273
tokens and now from that we have 121
distinct tokens
now suppose if we were to select the top
10
tokens with the highest frequency for
that i am assigning a new f test
underscore top 10 and i am using the
most underscore common function here and
passing 10 that is the number of items i
want
so
let's see the output
so as you can see
as i mentioned earlier comma appears 30
times that is the highest frequency of
any world and
is appeased five times so these are the
top ten words which are the most
reoccurring words in the given paragraph
or the given sentence
now let's use the blank line tokenizer
over the same string to tokenize a
paragraph with respect to a blank string
so as you can see we are importing the
blank line underscore tokenize
earlier what we did was import the word
underscore tokenize and now we are using
the blank line under stroke noise and
again we are passing the same ai
paragraph which i gave earlier and then
we are checking the length of the ai
underscore blank so as you can see it
provides us the output nine now what it
tells us is the number of paragraphs
which are separated by a new line in our
given document
so suppose you want to have a look at
the first paragraph or supposedly the
second paragraph
all you do is pass on the number and it
will give you the whole paragraph as you
want which is separated by a new line of
course
now coming back to our tokenization part
we have
diagrams diagrams and engrams now
diagrams are tokens of two consecutive
written words similarly diagrams are
referred to tokens of three consecutive
written words and usually n-grams is
referred to as tokens of any number of
consecutive written words for n numbers
so let's see how we can implement the
same using
nltk libraries for diagrams diagrams and
the engrams
so first what we need to do is import
backgrounds diagrams and engrams from
nltk.util
so let's take a string the best and the
most visual thing in the world cannot be
seen or even test
they must be filled with the heart what
a beautiful code so let us now first
create the tokens of the our string
using the word underscore tokenize as we
did earlier now to create a background
what we need to do is use the list
function and inside that we are going to
use the nltk.diagrams
and pass on the tokens
so as you can see it has created a
diagram of the given document similarly
if we create the trigrams and the
engrams
so all you need to do is change the
bigrams to trigrams and it will give you
the trigram list now let us now create
an n-gram list okay
so guys as you can see here we are using
the same function lltk.n grams and
inside that we are passing the quotes
under course token and
here in place of n we are going to give
our number so let's say suppose i
provide five here so as you can see it
has given us an n-gram of length five
now once we have got all the words or as
i say the tokens we need to make some
changes to the tokens and for that we
have stemming
now what stemming does is normalize the
words into its base root form so if we
have a look at the words here we have
affectation effects affections affected
affection and affecting now all of these
words originate from a single root world
that is the effect now stemming
algorithm works by cutting off the end
or the beginning of the word taking into
account a list of common prefixes and
suffixes that can be found in an
infected world now this indiscriminate
cutting can be successful in some
occasions but not always
and that is why we affirm that this
approach presents some limitations now
let's see how we can implement stemming
and we'll see what are the limitations
of stemming and how we can overcome them
now there are quite a few different
types of stemmer so let's start with the
portal stemmer so for that we are going
to use from nltk.stem import portastimo
and let's see what does this give us the
output of stemming the word having so as
you can see it has given us the root
word have
now similarly if we provide a list of
words such as give giving given and gave
and i have given this name of words to
stem
so forwards inverse system print words
and we are using the portastimer which
is the psd.stem method so as you can see
it has given us the output give give
given and gave so you can see the
stemmer remove the only ing and replace
it with e now there is another stemmer
which is known as the lancaster stemmer
so let's try to stem the same thing
using the lancast system and see what is
the difference
so let's stem the same thing using the
lancast system and see what are the
differences we have
so first of all we are going to import
the landcast system and we are going to
provide lst which is the lancast system
of function and in the similar manner
that we did for the portastema let us
execute the lancast system also so as
you can see here the stemmer has stemmed
all the words as a result of it you can
conclude
that the lancaster stemmer is more
aggressive than the potter stemmer now
the use of each of the stemmers depend
on the type of the task you want to
perform
now we have another stemmer which is
known as the snow world stimmer now for
snowball stimmer we have to provide the
language which we are using
so similarly if we use the snowball
stemmer on the given words
as you can see it has given us a
different output from the previous
lancaster stemmer so as you can see the
output is the same as that of the portal
stemmer here but it differs in different
terms now the use of each of this
stemmer depends on the type of the task
you want to perform for example if you
want to check how many times the word
giv is used above you can use the
lancaster's demo and for other purposes
you have the snowballs number and the
porter stem as well so what happens is
that sometimes
stemming does not work properly it does
not always result to give us the root
word so for example if we take the word
fish and fishes and fishing all of that
stems into fish so lemmetization is an
another concept
so on one hand where stemming usually
works by cutting off the end and the
beginning of the wood limitization takes
into consideration the morphological
analysis of the world so to do so it is
necessary to have a detailed dictionary
which the algorithm can look into to
link the form back to its lemma
now what limitization does is groups
together different infected forms of the
word called lemma it is somehow similar
to stemming as it maps several words
into one common root now one thing to
keep in consideration here is the output
of limitization is a proper word so for
example if we look back to our output
given by the lancaster's demo you can
see it has given us the output give
which is not a word so the output of
limitization is a proper word
and
for example if you take a limitation of
gone going it all goes into the word go
now again we can see how it works with
the same example of words let's try
limitization using the nltk library
so first of all we are going to import
the word net
as mentioned earlier it requires a
dictionary which the algorithm can look
into to link back the form into its
original lemma and since the output is a
perfect word so it needs to have
additionally so for here we are going to
import the wordnet dictionary and we are
going to import the word netlemortizer
so let us first
take the word corpora and see what its
lemma is
so what do you think guys what will be
the output
as you can see it has given the right
output which is the corpus
so let's use limitation on the given
words
so as you can see here the lemmatizer
has kept the words as it is this is
because we haven't assigned any pos tags
here and hence it has assumed all the
words as none now we'll learn about pos
later in this video but just to give you
a hint of what pos is pos is basically
parts of speech
so as to define which word is a noun
which is a pronoun
and which is a subject and much more
now do you know there are several words
in the english language such as
i at for begin gone no various
which are thought of as useful in the
formation of sentence and without it the
sentences won't even make sense
but these do not provide any help in nlp
so these lists of words are known as
stop words
so you might be confused as if are they
helpful or not
it is helpful in the english language
but it is not helpful in the language
processing so nltk has its own list of
stop word you can use the same by
importing it from nltk.corpus
so once we import the stopwatch we can
just have a look at the different
stoppers which is provided by nltk
so we need to mention which language we
are using so let's have a look at the
number of stoppers we have in english
format so just 179
so remember guys when we use the fdist
underscore top 10 to check the top 10
tokens which have the highest number of
frequency
so let us take that again
so as you can see here apart from the
word intelligence and intelligent
all of the words are usually stop words
or basically i should say that they do
not match any particular word they are
like special digits and characters which
do not add any value to the language
processing
and hence it can be removed
so first of all we will use the compile
from the re module to create a string
that matches any digit or the special
character now we'll create an empty list
and append the words without any
punctuation into the list
so i'm naming this as post punctuation
and if you have a look at the output of
the post punctuation
so as you can see
it has removed all the various numbers
and digits and the comma and the
different elements
now when i was talking about pos which
is parts of speech
now generally speaking the grammatical
type of the word
the verb the noun adjective adverb and
article now it indicates how a word
functions in meaning as well as
grammatically within the sentence
a word can have more than one parts of
speech based on the context in which it
is used for example if we take the
sentence google something on the
internet
now here google is used as a verb
although it is a proper noun now these
are some sort of ambiguities and the
difficulties which makes the natural
language understanding even much more
harder as compared to natural language
generation because once you understand
the language generation is pretty easy
so pos tags are usually used to describe
whether the word is a noun
an adjective a pronoun
a proper noun singular plural verb is it
a symbol is it an adverb so as you can
see here we have so many tags and
descriptions
for the different kinds of words
all the way ranging from coordinating
conjunction to what work wrp
now let's take an example of these two
sentences so the first sentence is the
waiter clears the plates from the table
so as you can see from the starting if
we use the word take into consideration
the it is a determiner
now waiter is a noun cleared is a verb
the is again determiner the plates are
noun from is not defined here the is
again the determiner and the table is
again a noun
now again if we take into consideration
the sentence
the dog ate the cat
similarly this is the determiner dog
noun eight is the verb and again it's
the same determiner and the noun
so let's see how we can implement pos
tags and do the tagging using the nltk
library so let us take a sentence
timothy is a natural when it comes to
drawing first of all we'll tokenize it
using the word underscore tokenize
and then we are going to use the pos
underscore tag on all of these tokens
so as you can see it has defined timothy
as a noun is as a verb as a determiner
natural as an adjective when as a wrp
objective and it as a preposition it
comes as a verb to and drawing as a verb
as well
now let's take another example that john
is eating a delicious cake
so as you can see here the tiger has
tagged both the is
and eating as well because it has
considered is eating as a single term
now this is one of the small
shortcomings of the pos taggers when it
comes to tying the words
now another concept in natural language
processing is ner which is known as
named entity recognition
so the process of detecting the named
entities such as a movie a monetary
value be it an organization a location
quantities and a person from a text is
known as named intelligent recognition
now there are three phases of ner which
is first is the noun phrase
identification now this step deals with
extracting all the noun phrases from a
text using dependency passing and paths
of speech tagging which is the pos tag
now secondly we have the phrase
classification now this is the
classification step in which all the
extracted nouns and phrase are
classified into respective categories
such as location names and much more
now apart from this one can curate the
lookup tables and dictionaries by
combining information from the different
sources and finally we have the entity
disambiguation now sometimes it is
possible that the entities are
misclassified hence creating a
validation layer on top of the result is
very useful the use of knowledge graphs
can be exploited for this purpose now
the popular knowledge graphs are the
google knowledge craft the ibm watson
and also wikipedia
so if we have a look at a certain
example suppose this is the sentence the
google ceo sundar pichai introduced the
new pixel at minnesota roy center event
so as you can see
google is an organization sundar pichai
is a person minnesota is a location
henry center event is also an
organization this is an additional layer
on top of the pos tagging so as to
clarify and give us more depth into what
the sentence is about and what the
sentence is conveying us so for using
ner in python you need to import the any
underscore chunk from the nltk module in
python so once we have imported any
underscore chunk
now let us take this sentence into
consideration which is the us president
stays in the white house now again what
we need to do is tokenize this sentence
and after tokenizing what we need to do
is add the pos tags
so that it will be easier for us to
conclude the ners
so we are passing the any underscore
tags into any underscore chunks function
and let's see what's the output so as
you can see it has given d as a
determiner it has given us as an
organization and white house is clubbed
together as a single entity and is
considered as a facility so as you can
see adding a layer of dictionary and
then adding the tags
and then creating the name entity
recognition makes the understanding of
language so much more easier
now as you can see in the nei entity
list we have geosocial political group
geopolitical entity we have facility
location
organization and person so as you saw
just from the previous example
as you can see we have given
organization
we have facility
now an important thing to consider in
nlp is also
the syntax now any linguistics syntax is
the set of rules principles and the
processes that govern the structure of a
sentence in a given language the term
syntax is also used to refer to the
study of such principles and processes
so we have a certain rules
as to what part of sentence should come
up at what position now with these rules
we create a syntax tree whenever there
is a sentence as an input so syntax tree
in layman terms is basically a tree
representation of the syntactic
structure of sentences or strings now it
is a way of representing the syntax of a
programming language as a hierarchical
tree-like structure and this structure
is used for generating symbol tables for
compilers and later code generation the
tree represents all the constructs in
the language and their subsequent rules
now consider the statement the cat sat
on a mat
so as you can see this is how a syntax
trade looks like
we have a sentence we have the noun
phrase the preposition phrase and again
the noun phrase is divided into article
and noun then we have the verb which is
sat again we have the preposition which
is on and again finally we have the noun
phrase which consists of article and the
noun now in order to render syntax trees
in your notebook you need to install
ghost strips which is a rendering engine
so you can download ghost strip from
this downloads page
i'll not go much into the details of
that
now let us discuss
an important concept with respect to
analyzing the sentence structure which
is chunking now chunking basically means
picking up individual pieces of
information and grouping them into
bigger pieces
so as we saw earlier that we have a
sentence and we divided it into
different tokens that was tokenization
and this is sort of like the opposite
part or we should say the opposite of
what we call known as tokenization a
little bit of changes to that
tokenization part we'll see what are the
different changes now the bigger pieces
are usually known as chunks here now in
the context of nlp chunking means
grouping of words or the tokens into
chunks now let's have a look at the
example of chunking here
so let's consider the statement we got
the pink panther
so as you can see here the pink which is
an adjective panther which is a noun and
though which is the determiner are
chunked together into a noun phrase
now this also helps in determining and
the processing of the language
so suppose if someone is asking whom did
we caught this morning so the regular
response to this question would be we
caught the pink panther now the question
is asking who so the pink panther
becomes a noun phrase basically so this
is something what the chunking does is
that it understands the language and
when it has understood what it does is
basically picks up the individual pieces
of information and groups them into
chunks
so that it will be easier for us to
process that data
so let's take a new sentence the big cat
ate little mouse who was
after the fresh cheese
so let's tokenize it and add the post
tags also so as you can see just under
one line or i should say and one command
i have tokenized it and i have added the
pos tags also
so let's define the grammar here
so next what we are going to do is
create a regular expression parser for
the grammar np which is the grammar
which we have provided
and then we'll create the chunk
underscore result and we'll pass this
whole sentence or i should say the
tokens into the chunks
there was a slider as nltk was unable to
file the js file but as you can see here
we have a tree in which the np is
the big cat
the big cat is a noun phrase it is a
verb tree then again we have the noun
phrase which is the little mouse who is
a word phrase was is a verb after is in
and again we have the noun phrase the
fresh cheese so you can see here
although we do not have a particular
tree like structure we can see that from
the outer we can see we have a tree
starting from the start is here and the
end is here and you can see it starts
with the noun phrase the big cat now the
big cat as i mentioned earlier in our
previous example similarly to the pink
panther it has been taken into
consideration as a noun phrase which is
basically a chunk it has created
different chunks in the given sentence i
should say
so this is it guys i hope you got an
idea of what the different parts of nlp
is what is nlp how it is used
the different elements of nlp such as
starting from tokenization stop words
stemming limitization again we have the
stopwatch part of speech pos tags ner
named entity recognition and after ner
we had the syntax
we created a syntax tree to know how
exactly the sentence is arranged and how
it uses the dictionary the pos tags and
the tokenization to create a particular
tree so as to form a meaning out of that
sentence
and finally we are using chunking as to
chunk together the small tokens into
meaningful words so guys i hope you
liked this session and i'm sure you guys
will start implementing these procedures
on the given text using the nltk library
and let me tell you one thing guys the
nltk library is filled with lots and
lots of text data and lots and lots of
example this is just the beginning of it
and you can dig deeper into nlp and go
into topics like context free grammar
and much more which is already there in
the nltk package and is very useful when
it comes to natural language processing
thank you