hello everyone welcome to the semicolon
in this tutorial is going to learn word
embeddings now word embeddings are a
method of extracting features out of
text so that we can input those features
into a machine learning model to work
with text data in machine learning and
as you all know the internet is filled
with text. so these methods play an
important role in lot of stuff so word
embeddings try to preserve syntactical
information. now the previous methods of
word embeddings which we have seen
sorry the feature extraction with text
which we have seen were count vectorizer
and tf-idf vectorizer now these two
methods relied on the word count in a
sentence but they didn't save any
syntactical information as such. so word
embeddings is one such method which
tries to do it. so let's see how it works
every word has a vector so a cow a
rat a cat each one will have a vector
now initially what we do is assign
random vectors or one hot vector now one hot
vector is a representation where only
one bit in a vector is zero say there
are 10,000 words in our corpus so the
our vector length will be 10,000 and
first word will have first bit as one
rest all 0, the second one will have 0
second bit 1 and rest all 0. so we
initialize all the words like that and
then what we do is we iterate we take a
window size and then iterate throughout our
document. so when we do that there are
two models which come into picture, the
common ball of words and the Skipgram
so when we do that we try in the common
ball of words model, we try to fit the
neighboring words of the window to the
central word and in this SkipGram
model we try to make the central word
closer to the neighboring words so in
this process
after a lot of iterations through the
corpus we get trained vectors of each
word which preserves syntactical
information so these were some common
results after doing the training on
Wikipedia. so China was china - beijing
plus russia gave the vector of Moscow. so
this distance was preserved as you can
see so many of such not just one these
were these informations were preserved by
our model the word to vec is a word
embeddings model made by google. now this
is a very famous result of word to vec
that king - man + queen gives you women.
so there is something, I mean this
preserves such syntactical
information in word to vec. so let's see
let's take an example and see how it
does. so say the capital has vector 001
this spain has vector 010 and Madrid
has vector 100 these are initialized
vectors. now what happens is we try to
fit this to this, so we subtract the
error in this so our error would be 011
multiply learning rate and add it to
the spain. so now we have a new vector and
then we do the same thing with madrid
and then we'll have another vector and
we keep on doing this in a whole large
corpus. so what happens is finally
Spain and Madrid are closer now in
distance and even the capital becomes
closer to these three. now as we go on we
might get words Spain and madrid
together and that's why they tend to be
more closer. so this is how word vectors
is. now let's see how we implement that
in our in Python using the gensim
library. so now let's learn how we make
our own word to vec model using the
gensim library in Python. so you have to
import gensim and if you haven't
installed gensim just type Conda update
gensim in the Anaconda command prompt
and then you get it off. I have my corpus
in this directory, which is named as
jokes dot CSV now what this directory
contains is some question and funny
answers for it. so I'm not sure how our
model will turn out to be because of
this nature of our corpus but then
training our word to work
on Wikipedia would have been very costly
and it would take a lot of time for the
tutorial, so that is the only reason why
I haven't taken it but you can try and
measure the accuracy yourselves. so what
we do is we take the question and
answers and then combine it we could
have combined it before also then we
tokenize each and every word of it and
then we pass it to our model so
this is our gensim model now this is
all you need to do to create your own
model of word to vec. so the model dot
save is used for saving the model so
that you can use it later on as well
because doing this every time is a bit
costly then when you want to load the
model type this now if you want to check
the similarity of words then copy this
and ok we have to execute it first so I
am executing it and it might take a
little bit of time to execute because
the corpus is actually huge around
13,000 questions and answers so that
will take around 1 minute of time and
the Wikipedia thing might take around 20
minutes to run. so it would be
useless to train it just for practice
purposes, you can train it when you have
to do some serious stuff. so in this
gensim dot word to vec model we pass
the tokenized sentences the tokenized
corpus this represents the minimum count
of words so when the word is at least
once in the corpus then we includ it you
can mention it twice or thrice or how
many times ever you want the size is the
size of the vector so our vectors will
be 32 bit long so since this is now
executed let's type model dot most
similar and type the word. so here when
we type the word say hi we get the most
closest models closest words so
maybe hi son hi girl hi wife was
mentioned frequently in our corpus and
that is the reason I think it is like
that so safe we type mom so okay this is
bad but since this was a joke it might
be wrong choice of corpus so when we
type dad then we get all these words
near so buddy mom son and all that so
since our data was bad we can say that
it hasn't learned properly but if you
train it on word to vec the closest ones
would be very good and if you want to
check the vector of any model so
this should give you the vector of any
model so this is a 32-bit number and 32
bit vector, vector of length 32 so this
is what your word vector is you can pass
it through other machine learning models
and calculate many things anything
which you want you can cluster similar words
based on your distances very easily
using the k-means cluster or you could
do many other stuff. so that's it for word to vec,
if you like this tutorial hit the
like button if this helped you in any
way and you want to watch future videos
hit the subscribe button as well and
keeps sharing with your friends thank
you