In this short video we will talk about
tax representation in NLP and we don't
have any coding today.
In this playlist we had a video which
covered three techniques of doing NLP,
these three.
And we covered rules & heuristic a
little bit in our regular expression
video.
When you talk about the other two
approaches, the NLP pipeline looks
something like this. And in this we
already covered pre-processing by having
tutorials on stemming, limitation,
tokenization all of that in Spacey, so
far. Now we are jumping into the next big
topic, which is feature engineering. This
is one of the very important steps in
machine learning pipeline, and this is
the step where engineers or data
scientists spend a lot of time.
What exactly is features? Before we talk
about feature engineering, we want to
know what is features. So, if you don't
know the basic concepts about machine
learning, I'm going to give you a very
Layman understanding of features. Let's
say you're working on a property price
prediction problem
and trying to build a machine learning
model that can predict the price of a
property. Now when you go and buy home,
what are the factors that can impact
price. Well obviously things like area,
facilities, how old is the home, location.
These things are called features for
property price prediction problem. It's a
linear regression or statistical machine
learning problem. If you talk about dogs
and cats general image classification,
the way this works is how humans
recognize these images or in reality
let's say you're walking on a road you
see cat coming at you or dog coming at
you how do you know it's a dog and cat
and not a lion?
Right you in your brain there is some
detection happening. And the way it works
is if you think about properly is you
will detect individual features such as
nose. When you look at an animal and that
nose you will say yes it's a dog's nose
or cat's nose. Eyes right, so you look at
cat's eyes they're around. Dog's eyes are a
little different. Dogs ears are kind of
falling and cat has pointed ears.
Although some dogs also have pointed ears.
But you get an idea that every animal
has a distinct set of features. Whiskers
so, Eyes, Nose, whiskers yes are called
features, basically right. It's a general
common sense knowledge. And we use neural
networks for this image classification,
and I'm going to give you very
simplistic understanding of the neural
network, okay. This is for Layman
understanding, if you want to know
details, in YouTube search code Basics
convolutional neural network. I have
explained things in detail and in a more
accurate way. But this is just to give
you a general idea that, when you have a
neural network
each of these neurons are like units and
these units have a specific task. One
unit is saying whether the ears are of
cats ears or not. The other unit is
doing a task of identifying if, in the
image there is cat's nose or not? And if
you get yes answer for all all of this,
you can say yes. In the image, I have a
head of a cat. And if you have a head of
a cat and a body of a cat, it has to be
cat.
So this is how the neural network works
in layman terms.
Let's say if if I show you image like
this, you know I just use Microsoft Paint
and put some dogs body parts, as a human
when you look at it now,
you think it's cat's image because
little bit it looks like clay cat's
image. But you're not 100% sure. Why?
Because in your brain, your brain is
telling
these ears and nose they don't look like
cat's ears and nose. So, you will say no.
In your brain this is happening. You're
saying no no for certain features.
And then if I tell you let's say, if I
remove everything and if I just show you
the head of of this animal in the image,
you and I if I ask you is it cat's head?
You say no, it is not cat's head because
the ears and eyes are kind of looking
different. They are not cat's ears and
nose, actually.
Hence, your overall answer might be no.
Now we talked about images and uh, the
property price prediction problem which
was more like statistical machine
learning, this is more like convolutional
neural network.
How about text? Let's say you have this
text. I have I've given you three words:
Dhoni, Cummins, Australia. Now if you're
following Cricket you know that the
first two words are actually cricket
players. The third word is a country
called Australia.
How do you represent this text into
features? You know, how do you extract
features out of it? Because previously
see for for the image, ears, nose, eyes
they are features. For property price
prediction, area, location, facilities,
bedrooms, these are the features. So what
are the features for word right it seems
little bit tricky
one way could be
you can ask couple of questions you can
say okay is this a person is that is he
healthy or fit
is it a location? See I have location, I
have zero value means Dhoni is not a
location. It's a person so the value of
location is zero, and person is one. But
Australia is a location so value is one,
and person is zero. And these are like
handcrafted features, all right. So these,
are the features that represent these
words. And I can have a feature vector.
So this is one of the common techniques
in natural language processing, where
you will present a word as a vector, set
of numbers instead of one number. There
is a reason you don't want to assign
just one single number. If you present it
as a vector, you can do mathematical
operations such as cosine similarity. If
you don't know what is cosine similarity,
again go to YouTube search code Basics
cosine similarity. I have a math and
statistics playlist just for machine
learning you will get your concept clear.
I would highly recommend actually, you
watch that playlist. You know, in YouTube
type code basics
or let me maybe just show you. So, in the
YouTube
let's say if you type in Code Basics
maths statistics,
you will find this particular playlist.
My wife likes to see CID by the way. So
uh, here you will see
cosine similarity.
And this cosine similarly this Vector
mathematics, is very helpful in NLP. That
is one of the reason why we present, uh
why we present the words as vectors,
all right.
So essentially what we did is we had a
word
we use some technique. Now let's not go
into detail. You will be like okay how
did you come up with these questions?
I just made it up folks for your
understanding, okay. As we go,
uh as we make progress in this course,
you will find out. But just for your
understanding, we use some criteria to
come up with a vector for a word. And you
know, there is a benefit of doing it this
way. Because now when you're comparing
Dhoni versus Cummins,
you will see lot of similarities. See 1
1 0.9 0.87 So when
you do a cosine similarity between these
two words, Dhoni and Cummins will be more
similar, than Dhoni and Australia. The
cosine similarity between Dhoni and
Australia will not be they won't these
two vectors won't be that similar.
But the vectors of Dhoni and Cummins
Cummins, vectors is basically these
numbers, you know. It's like a basic math
if you, you need to know basic math,
Vector math. And for that, I I refer you
to that playlist. So the cosine
similarity between Dhoni and Cummins is
higher. They are highly similar. So now
when you are doing any kind of NLP when
you're working on NLP problem,
having similar representation for
similar words is very very useful,
correct? You are building sentiment
analysis model, and you have a product
review, let's say where you say the
product is worse, okay?
And you have built a model based on this
kind of training set, and let's say when
you're predicting and you say if the
product is bad. Now worse and bad are
similar words. So if you have this kind
of vectors, it will help you figure out
that these are similar.
So, here the idea is you converted a word
into a vector of numbers. Why do we do
that? Machine learning models do not
understand text, that's the fundamental
thing you need to remember. Machine
learning models need numbers they can't
work on text. You have to convert your
text into numbers. And this is one of the
ways of converting text into a number,
which is a vector representation. It is
also called Vector space model.
So feature engineering is basically a
process of extracting features from raw
data. So you had raw data which was Dhoni.
You extracted all these features you
know, person, healthy, fit. You
constructed this vector, and this process
is called feature engineering. In NLP it
is also called tax representation. You
represented this text as a vector of
numbers. It is very simple folks it's not
a rocket science at all.
And this model this mathematical model,
of coming up with a vector, for a text,
and when I say text, it can be word,
phrase,sentence, paragraph. These are
different text units. When you come up
with a vector for any of these text
units, it is called Vector space model.
Now there are various approaches of uh,
coming up with a vector for a text, or
there are various approaches of doing
text representation in NLP,\.
One Hot encoding, bag of words, I have listed
few here one hot encoding is not very
very popular. We have looked into this in
our
machine learning playlist. If you watch
Code Basics, Again YouTube if you search
Code Basics machine learning, uh you will
find my playlist. Uh, in the next video I
will tell you how many videos you need
to watch.
Uh, but in the next video we are going to
look at most likely bag of words. And
we'll do coding, we'll do some a
classical problem of email spam
detection. That's the agenda for the next
tutorial. But in this tutorial I just
wanted to give you a brief idea on what
is text representation. And I want to end
this session with a very important and
insightful code from this book, Practical
NLP okay? I think you you should read
this book. This is
a very good book in NLP. The book says
often in NLP feeding a good
representation to an ordinary algorithm
will get you much farther than applying
top-notch algorithm to an ordinary text
representation file. Now you understand
as a data scientist or NLP engineer, you
should not be obsessed about
what exact machine learning model you
are using. You should put more focus on
the tax representation.
See, if you can extract the features in a
better way from the text, and if you can
represent text in a good way using
Vector space model, it is going to take
you very very far. I hope you liked this
session. If you did, give it a thumbs up.
If you have any question Post in a
comments box below. In the next video
we'll do coding, we'll do emails spam
detection,
um and we'll use a bag of words.
[Music]