i hope you have seen my previous video
on what is bert in this video explain
how bert works
the fundamentals of it in today's video
we are going to
do email classification whether it's a
spam or non-spam using bird
now bird will convert an email
sentence you know the the whole email
into
an embedding vector so we saw in a
previous video that
the purpose of bird is to generate
if is embedding vector for the entire
sentence
and that is something that we can feed
into our neural network and do the
training
so here we will generate a vector of 768
length y 768 we have covered that in a
previous video
and then we will supply that to
a very simple neural network with only
one dense
layer one neuron in the dense layer
as an output will also put a dropout
layer
in between just to tackle the
overfitting
now if you open this bird box by the way
it has two components
pre-processing and encoding and we
talked about that in a previous video as
well so previous video watching that
is quite a prerequisite so let's jump
into
coding now now here i have downloaded
this file from kegel
simple two columns category hammer spam
and here is the content of your email i
have imported few basic
libraries here in my jupyter notebook
and i'm going to simply read this csv
file into my pandas data frame which
looks like that
and then i will do some basic analysis
you know i will do df
group by let's say category
so here i have four eight two five ham
emails and 747
spam emails you can clearly see
there is some imbalance in our data set
so we need to take care of that but
before we do that
uh we will create a new column in my
data frame
you know we'll call it spam so let's
create a new column
and if the value spam the value of
this spam column will be 1 ham it will
be 0
and you all know if you want to create a
new column in a data frame from an
existing column
you can use apply function and that will
take
lambda and what we you are doing is if x
is equal to spam then the value is one
you see this is how ternary operator in
python works
else value is zero
and now if you do df dot head see we
simply create a new
column zero one say spam one
hem zero all right so far
so good now let's do train test split
so i'm going to use our standard
sql entry in this split function and in
that
my x is actually a message
and my y is the spam
okay the spam column and i'm going to
store the output into
these variables this is pretty much a
standard practice
in machine learning world and okay i
will do
our test size to be
point two so eighty percent training
samples
twenty percent um taste sample
let let me check how it split the
spam and non-spam
so value counts
okay
so i'm checking this to make sure there
is a balance
okay so let's see okay 149
divided by 967 okay around 15 percent
spam in test
and 3859
okay so it is good balance but still
to be on a safe side i will
say stratify so when you do stratify it
will make sure
there is a balance you know
it's not like in your training data set
if all the samples are zero
and there are lesser two samples which
has spam value
then model will not be good in terms of
detecting the spam
okay so that's why i supply this
stratify i mean before stratify also it
already did a good job
but this is just to be on a safe side
now comes the most interesting part
which is creating the embedding using
bird
okay so how do you do that so for that
you have to go to this tensorflow hub
website
and click on text and go to bird model
now in but we are going to use this
first model so we saw in a previous
video that
there is an encoder and there is
pre-processing step so first you do
pre-processing
so you click here you copy the url
okay so this is my pre-process url all
right
and you go back
you go to text bird and
you go here and copy this url this is
your
main encoder okay so this is your
pre-processing url
and this is your main encoder
url so i'm going to use
keras layer hub keras layer basically
okay and call that bird pre-process
and then i will use the same hope keras
layer here
and call it board encoder
see we saw in a presentation there are
two steps invert encoding pre-processing
and encoding so that's what
we did exactly
okay when you run it it's gonna take
some time because
it is downloading the bird model you
know it's
somewhat around 300 megabytes so based
on your internet speed it might take
some time
but essentially you are downloading a
train
model which is trained on all the
wikipedia
and book corpus so now in our task
we'll be just directly using that train
model to generate the embedding vectors
after model is downloaded i am going to
define a simple function
that takes couple of
sentences as an input and returns me an
embedding vector so basically the way
i'll use this function is okay
supply an array of sentences
and any sentence okay and that should
return me the embedding
vector for the entire sentence
and if you've seen again my previous
video
this pre-process handle that you get you
can use it as a normal function
and you supply your sentences here and
it should return you the pre-process
text so i will just call it pre-process
text and then
use the birth encoder okay and when you
use the about encoder it returns a
dictionary out of which
you need to use pulled out pull output
is basically
the encoding for the entire sentence
again if you want to know
what other elements are there in the
dictionary you need to watch my previous
video it's sort of like a prerequisite
all right now see when i run this
it is generating for this sentence this
is my
embedding vector the size is 768
for this second sentence this is my
embedding vector so
we have achieved the major goal here
which is
generating the vector you know vector
using the bird and i just give you a
simple function but in reality
we will be using tensorflow layers
okay but before we go there let me
generate some embedding vectors so
for some more you know words let's just
generate it for words let's say banana i
want to
see what kind of embedding vectors it
generates
for a couple of fruits and then
you know what i will compare the fruits
with jeff bezos and bill gates
so these three are people these three
are fruits so
let's see what kind of embedding vector
it generates
and now i have all these embedding
vectors right so if you do e
6 by 7 68 okay i am going to use
cosine similarity so if you have seen my
cosine similarity video
if you do cosine similarity you will
find this video where i have explained
you know what is exactly cosine
similarity
so if you don't know watch it it is used
to compare
two vectors so here i will compare
let's say banana banana's embedding
vector
with uh grapes embedding vector now
this takes a two dimensional array so
i'm just going to wrap it up in a
two dimensional array okay you see 0.99
so if it is near to 1 it means
these two vectors are very similar so
banana and grapes are similar because
they are fruits banana and
mango is also similar because they are
fruit but let me compare banana with
jeff bezos it's kind of weird right
comparing banana with jeff bezos
see 0.84 but still they're not 0.99
they're not as similar as banana and
grapes
okay and by the way you have to use this
with a
with a little caution i mean cosine
similarity
is not exact vector similarity okay so
sometimes you might see some unexpected
result but that's okay
now let me compare jeff bezos with elon
musk
say again 0.98 so you get the point
behind
bird now okay now let's build a model
so far in our deep learning series we
have
built tensorflow models using sequential
model okay we are going to now use
functional models so there are two type
of model sequential
sequential and functional
okay so what is the difference between
the two
i'm going to link a good article here so
in the sequential model you add layers
one by one as a sequence you see
but in a functional model you create
your input
then you create a hidden layer let's say
and supply input as a
function argument then you create hidden
one then you supply that into
hidden two's argument and so on and then
you create model using
inputs and outputs now this allows you
to
create a model which might have multiple
inputs multiple outputs
like something like rest net you know
you can also
uh share network layers with other
other models so there are there are some
differences you read the article you
will get an idea
so here i'm going to build a
functional model okay
so the first step is you create your
input layer
the shape is going to be this because
the sentence length is
varying and
my data type is string and
name of this layer i will call it text
or
input whatever you know you can give it
the name that you like the most and this
will be my input layer
then we are going to do this
these two things
so here i supply input okay
same thing and then
uh bird encoder and bird encoder
takes let me just do output here
so outputs
okay and from the outputs i get pulled
output so pull output will be
the sentence encoding so pool output
will be this
okay this i need to
i will create one dropout layer which i
have not shown in the picture
so let me feed that pulled output into a
dropout layer
and then last layer will be one neuron
dense layer okay
so let's create dropout layer here
dropout layer is used to tackle
the overfitting
sometimes even if you don't do it it's
okay it helps
and in that dropout layer
you pass this as an input okay so now
i'm going to drop 10 of neuron
okay and i will call it this
dropout and let's call it
l l is the layer and the second one
is the dense layer with one neuron
and since it's a binary classification
you know like one zero
kind of thing i will do
sigmoid and the name of this layer is
output and again we are using functional
api so
you need to treat this as like a
function and pass in the previous layer
here
and then i will say overwrite the same
variable you know
okay and then in the end my model
is nothing but this model
which has two parameters inputs
outputs now the inputs will be
this so it's an array you can supply
multiple inputs as well
so here this is the input
and output will be l okay
and you can do model dot summary
okay output is not defined
so output
outputs great
now here my trainable parameters are 769
because
i have 768 neuron here and this one so
total
769 my non-trainable parameters are so
much so these are
the parameters from my birth model bird
is already trained
so i don't need to train them again and
when you are doing you know model
building you know that you do model
compile
where optimizer loss these are like
pretty much standard
things that we use in all our tutorials
loss is binary cross entropy because we
are doing binary classification here
and then i'm going to now run the
training so model dot fit
x train y train
epochs let's do tan epochs now
this is gonna take time because the
whole encoding process is little slow
and we have so many samples so based on
your computer it might take time i have
a powerful
computer and gpu but still takes few
minutes so you to be little passion
you can reduce epochs if you want okay
so i reduced epochs to five and i got
ninety three percent accuracy
then i do model dot evaluate on my extra
sweaters
i got ninety five percent accuracy which
is which is so
good actually so now i do inference so i
have a couple of
uh emails actually it's not reviews it's
emails
and on that email when i do predict uh
see the first three emails are spam
the second the rest of them are not spam
they're legit emails
and in sigmoid whenever the value is
more than 0.5 it means it's a spam
and it is less than 0.5 it means it's
not a spam so you see
so these things worked out really well
for us
all right so this tutorial provided you
a very
like a simple explanation of how you can
do
text classification using bird you can
use
bird for variety of other problems as
well just
such as movie review classification or
name entity recognization and by the way
i i have an exercise for you and the
exercise is actually
very simple you have to just do copy
paste so
go to google red tensorflow tutorial
and in that go to text tutorial and look
at classified text with
bird so what you need to do is
you need to just run this code on your
computer so just
copy paste these these lines you know
step by step in your notebook and
just run it and try to understand it
this
tutorial is similar to what we did but
the
data set is much bigger they are using
tensorflow data set api so
in terms of api also they are using
little different we use pandas
and they are also using some caching
the model is also little different so if
you practice this
you will consolidate your learning
from this particular video so i hope
you're you're going to practices i trust
you all you're a census student
so please open a notebook copy
paste these lines one by one try to
understand it
and see how it works if you are
confident
you can just load the data set and
finish rest of the tutorials by
referring to this page
but without referring to this page okay
so thank you very much for watching
i will see you in the next video if you
like this particular video
give it a thumbs up and share it with
your friends