In the last video, we looked at how you
can convert text into numbers so that
machine learning models can consume that
text.
In this video we are going to look at
two of those approaches which is
label encoding
and one hot encoding. Now before we
continue with the discussion,
uh I would suggest that you follow some
prerequisites because in the coming
videos we will be doing some coding and
you need to have basic knowledge on
pandas.
Now I have pandas tutorial playlist. Go
to YouTube, search for codebasics pandas,
you find my tutorial playlist very
popular,
uh and you need to follow first 6 or
7 videos just to get a basic
understanding of pandas.
After that,
you need to know some machine learning
fundamentals too. Again I have a very
popular playlist on YouTube codebasics
machine learning playlist.
In this,
follow first maybe 8 or 10 videos,
and then you can resume this series.
We're going to talk about
spam detection. So spam detection is a
classical problem, uh for text
classification. Again, text classification
falls under
Natural Language Processing domain,
and this is the screenshot from my
gmail account.
You all know
gmail very accurately detects the spam
and saves us a lot of headache.
So how do they do that? They use machine
learning behind the scene.
For example, this is again a real example
of a spam email,
where they will use lucrative offers.
They will say
I have
55 million dollar in in in some bank
account,
and they will use this
urgency,
you know urgent business assistance, that
those kind of keywords.
So how do they classify the text into
spam and ham? Ham means not a spam.
When you have a text, a typical approach
is
to convert that text into a vector,
and this is also called
text representation
or,
feature engineering. Basically machine
learning models,
they understand numbers. They don't
understand text. That's the reason
you need to convert text into numbers,
and these numbers are often
a vector and this is called vector space
model.
And you can then feed this to
a machine learning model such as Naive
Bayes classifier. Naive bayes classifier
is one type of machine learning model,
and that can classify text into spam or
no spam.
Again if you have followed that tutorial
playlist which I talked about, my machine
learning tutorial playlist, you will have
some understanding of what I'm talking
about.
So this is the
overall approach. So by now you're clear
that, whenever you're solving
NLP problem using machine learning, you
have to convert text
into
number or a vector.
Let's say I have
these sample emails: the first and third
email are
a regular email. The second email is spam.
Let's discuss
how we can convert text into numbers. So
pause this video for a moment, and just
think about some approach. I think you
will come up with some basic approach on
your own. You don't need any guidance
from any book or my video.
Just think about it
for a second.
So one approach
that we can come up with very very basic
approach is,
let's create
a vocabulary out of all the words.
So
let's say I have 100 email in my
database, you know and I'm using those
100 emails to train my machine learning
model.
I can take
every word
from those emails,
maybe apply some
stemming, lemmatization,
uh convert them into some base words,
and build a vocabulary. Vocabulary is
basically collection of all the words in
your emails,
and the vocabulary
may look like this:
I have 3 dots means let's say I have
few thousand
words in my vocabulary, and how did I
convert the, how did I generate these
words? Well I
went over
all the emails in my training dataset,
and took each word.
And each unique word is you know one
word in the vocabulary.
And let's say I assign numbers to these
words. Let's say I sort them alphabetically,
and and I assign
these numbers.
So number for add is 1, auto is 2 and
so on.
And then
when you take any email,
you're just doing a simple lookup.
So let's say,
the
number for hey is 12.
You see hey is 12,
Pranav is 187 it's not visible somewhere
hidden in 3 dots.
Can is 7,
you see can is 7
and so on.
You can create a simple vector, or a list
you know if you're using Python
terminology, is vector is
nothing but a
list of numbers.
You can create a simple vector,
to represent that text and this is a
very primitive way of representing text.
As a vector it is called label encoding.
You might have encountered this term,
label encoding if you have done
machine learning.
This is a very basic approach.
All right! Let's look at the second
approach okay?
Second approach is
all these words are part of my
vocabulary okay? Add, auto and
and 3 dot means,
let's say I have 2000 words in
my vocabulary.
You put them like this you know at the
top,
and then
you take your actual text that you want
to convert into a vector. So let's say
this is my email:
Hey Pranav can you add auto whatever,
and then for each
word,
what you will do is
you will create this kind of vector,
where the position of that word will
have 1 and remaining words will have
0. For example I have can here. So you
see can here, so can is 1.
Other words are 0.
Similarly hey and Pranav are kind of
hidden in these three dots. I don't have
enough space on my screen. So I can't
show you the entire vocabulary. But you
get the point.
See here
add. So if you have a word add,
the position for add is the first
position, that will be 1
Remaining position is 0, and this is
called
one hot encoding.
Again if you have learned
machine learning, you would know about
one hotend coding because it's a popular
technique
used to represent categorical variables
in machine learning,
okay? So again,
not a rocket science there is very
simple concept.
That you have your entire vocabulary
and,
wherever the you assign an index in that
list, for each individual word, and
wherever that word appears,
you just put 1 at that position,
remaining positions are 0.
So we looked into
two approaches: label encoding and one
hot encoding
In NLP, people don't use these approaches.
I just wanted to describe these
approaches,
so that you at least know
that these are primitive ways of
representing text
into a vector.
There are certain disadvantages of these
two approaches, and that's due to those
disadvantages they are not
used much in machine learning nowadays.
In machine learning nowadays, they use
most of the time word embedding, TF-IDF,
bag of words okay?
So let's look at the disadvantages of
these two approaches.
Let's say I have a sentence
called: I need help
and this is my
one hot encoded vector.
Let's say I train my model,
and then
for prediction I get a sentence, like "I
need assistance." Now help and assistance
are kind of similar word. So if i have a
similar
representation of these two words, it
will help me.
But here see
this one hot encoding is kind of dumb
because
it just looks at the position of a word
and puts 1 there. So if you look at
these two vectors, help and assistance
they're very different.
See and help I have all these 0s and 1
is at this position. Assistance again
I have all these 0s and
assistance, you know here I have 1.
Just by looking at these two vectors, you
can't say that they are
similar words.
So one hot encoding doesn't capture the
meaning of the word in an accurate way.
So that's
the disadvantage number one.
The second disadvantage is,
let's say in my vocabulary I have 100
000 words in that case you you would
have already sensed
that each word
will will occupy the size of 100k. You
know the
vector for each word would be 100k.
That's too much memory,
and that is just one word.
What if I have an email
that has 500 words in it.
So now you you multiply 500 with 100k
and that much memory I need just to
represent one email into a vector.
So it consumes not only too much memory,
but when you are doing machine learning,
it will
waste a lot of compute resources.
So that's the disadvantage number two.
The third
disadvantage is,
let's say
I
have this vocabulary. I train my model,
and when I got the actual email for
prediction okay, I completed my training,
and when
I'm getting an email for prediction,
let's say it has this word called
Bahubali, which
which is not part of my vocabulary.
So how do you represent this,
uh into one hot encoding vector?
Well you can have
undefined word as a last, you know you
can have the last
word here in the vocabulary could be
undefined or out of vocabulary, you can
put it there 1 there. But then every
new word that you are encountering, will
have same
numeric representation.
So you see that this is not accurate,
and it suffers from out of vocabulary
problem.
What if I had a
numeric representation
which can even represent the unknown
words? You know the words that it has not
seen before in an accurate way. That
would be cool,
but
clearly label or one hot encoding
doesn't do this.
And by the the last memory problem that
I talked about applies to one hot
encoding, not label encoding. But
see these two approaches are dumb.
Basically label encoding, one hot
encoding are
dumb way of representing text into
numbers.
Let's move on to the next disadvantage.
Let's say I have a sentence, "I need
banana"
and I converted I I created one hot
encoded vector for each of these words,
and I want to feed this into machine
learning model.
I can flatten these 3 vectors and
create a single list. Let's say each
vector is thousand
numbers each, I can flatten the list.
Flatten means just join this three list,
I can create one list whose size is
3000,
Feed it into machine learning model.
Okay that's fine.
But
see all my emails are not of the same
size. So my next email could be, "I need
banana shake."
In that case the size of the flattened
vector would be 4000 and
machine learning model expects similar
size. They see when you're defining
machine learning model, let's say you are
building a neural network.
Your input layer of neuron has to be
fixed size. It cannot change,
you know I can't feed 3000
one time, 4000 second time. It
has to be same. So we need to come up
with a numeric representation of a given
sentence in such a way that, for all the
emails or for all the text, I get a fixed
size vector.
One hot encoding doesn't
give me that,
and that's the disadvantage number four.
All right?
So that's all I have for this session. Uh
I just wanted to quickly cover one hot
encoding and label encoding. These are
very primitive ways of doing text
representation,
and these approaches are not used in
machine learning or NLP because of the
4 disadvantages that
I mentioned in this video. In the next
video, we are going to look into
bag of words and we will do some coding,
don't worry! I know we have been having
few videos where we did not do coding.
So we are going to do some coding! Also
for last few videos, I have added
exercises.
So
whenever you're watching my video in
general,
uh
make sure you check out a video
description, uh because sometimes you
know
after publishing a video, I might add
some exercises. So it's always beneficial
that
you check the video description. If you
like this,
please give it a thumbs up, share it with
your friends.
You can spread the word, I'm putting a
lot of effort in making these videos.
So if you find them useful,
share it on LinkedIn. LinkedIn is a very
good platform where people are serious
about learning and about job, etc.
So you can maybe add a post in LinkedIn
and talk about
uh your learning.
All right? Thank you!
[Music]