hey everyone welcome to the semicolon, in this video we're going to train a word2vec model
with the reddit World News data set using gensim. We're going to see how to switch between skipgram
and continuous bag of words (cbow) for training the word2vec model. We will also try using a pre-trained
word2vec model and then we'll spend some time playing with the model trying out the
famous results. So let's start. The first thing you need to do is install gensim type pip install
gensim and if you're using Windows type that in your anaconda command prompt while this installs
let's open the anaconda navigator to use the jupiter notebook. Gensim is installed now and
this is the Anaconda navigator let's start the Jupiter notebook and create a Python 3 notebook
we need to import word2vec for training the word2vec model and keyedVectors for using
a pre-trained word2Vec model let's import pandas and nltk as well. The redditWorldNews.csv
file has the news headlines posted on the subreddit world news we're going to use those
headlines to train our word2vec model as I couldn't find a smaller and better data set
for this purpose feel free to use any other data set based on your purpose we are only interested
in the headlines so let's just take the title column we need an NLTK to tokenize the words
and if you are doing it for the first time, you might have to download some additional information from NLTK by
using nltk.download() like this. Now let's tokenize the words and have separate array of
words for each sentence and this not necessary gensim will convert it into free text anyway
you may as well pass just the words, when the corpus is large this takes quite a bit of time
so as this goes on let's let's write the code for actual training so we use the word2vec method
we pass text the minimum word count for a word to be considered for converting into vector and
the size of each vector and this doesn't execute because we need to pass min count not count and
executing it start the training and... it's now trained using model.most_similar() helps us
find the ten closest word in the vector space we created. So if you see the most similar words for
man were woman girl boy so on. So remember this is a headlines corpus, we don't expect it to have a
very good accuracy but this did decent. Let's try the famous relationship King - man + woman and
it should be queen but it's not it's king and this weird and I think it's due to the quality
of the dataset anyway let's move on. To look at the vector of the word man we can use model["man"]
and this returns the vector and this how man is represented in our vector-space
well these methods seem to be deprecated so let's use the methods in the current version
and now these warnings are gone. So was this model skipgram or continuous bag of words(cbow) ? let's
look at the documentation to understand this so this is the word2vec documentation you'll
find APIs, usages and examples here make sure you visit it. Ff you go down and try finding parameters
for trainables you can see that with min_count and the size parameter which we have
been using we also can pass the SG which is short for skipgram when SG is set to 1 it's skip gram
otherwise it's continuous bag of words(cbow) so our model was continuous back of words and you know what to
do when you want to switch it to skipgram and there are so many parameters you can tweak like
the window_size, the cbow_mean the vocabulary size to help with limited RAM if you don't know
what these parameters mean look at the previous video which explains the word2vec and this how
we train the word to work model with any corpus and different parameters and different algorithms
so now let's see how to use a pre-trained model let me restart the kernel so that it's a little easy
on my ram let's go to the word2vec project by Google here you have the C++ code for
word2vec on which gensim is based having a C compiler installed speeds the process by around 70 X as
claimed by the gensim official website so in this Google word2vec project you'll also find
a pre-trained model it's 1.5 gb and i've already downloaded it and extracted it in the same folder
so we just need to import it now the method used for importing is load_word2vec_format() and we
need to pass the filename as parameter and set binary flag as true I'm limiting the number of
words two hundred thousand words this we'll just consider the hundred thousand most frequent words
making the import faster and... the file name is wrong let me edit it and oh so quick that's cool
so now we have imported the model which was pre trained. Let's play with it let's find the most
similar words to man again yeah and it's almost the same result let's look at the vector of man
now this is huge the model we trained we said this size to 32 and here it's 300 a huge vector why is
it giving us the deprecated warning though? isn't it weird, we use wv to avoid the deprecated
warning well it turns out that WV is deprecated on keyedVector and the normal way is deprecated
on word2vec. Little weird, no problem, that's gone let's play around with this model let's
try some famous results of word2vec. Say... King - man + woman is... the first result is king
but the second result is Queen which is right the accuracy is affected by just using 100,000
words maybe let's try something else Germany - Berlin + France is France. Is this model wrong?
oh no it's Germany - Berlin + Paris and that is France yeah we were subtracting a capital adding
a capital so that is right let's try something else messi - football + cricket messi is not in
the vocabulary let's try M with uppercase and it's a list of all famous cricketers wow! never
thought this would work let's try some other sport hmm... tennis and Nadal, this is great so this how we
train a word2vec model in gensim and this is how we use a pre-trained model and now you can use these
vectors as inputs to various neural networks in the next tutorials we'll pick up a simple use case
for different forms of data with deep learning like videos music text and any other if we find it so
hit the bell icon and subscribe if you want notifications when that goes live ! thank you