what is going on everybody and welcome
to part 7 of our chat bot with python
tensorflow and deep learning tutorial
series in this video what we're doing is
actually one deploying the model but to
talking about a real high-level sense at
least two major kind of model frameworks
that I've personally used for chatbots
then we'll go ahead and deploy a model
and then we'll talk about some more of
the intricacies of how those models work
and what you can kind of do to tweak
them so first of all when I have like
first started looking into doing chat
BOTS it was actually kind of challenging
to figure out like okay well okay let's
say you want to do deep learning and you
don't want to do a rule-based chat but
like what do you even use to do that
right and there there was a lot there
was so much information out there for
doing like a rule-based chat bot most
likely because the most popular and the
most successful chat BOTS really to date
have been rule-based at least to some
degree arguably the most successful ones
right now are kind of like mixtures of
the two like have rules and have AI and
as we'll see here and at least all the
models I've had have almost always
needed to append some rules to the
output if not to stop the AI just
repeating itself but also just to stop
responses that just never make sense or
whatever so chances are you're almost
always going to need a combo of the two
but maybe in a few years we'll be able
to make it advanced enough chat bots
that don't anyway as I was looking I
stumbled into sequence the sequence
stuff with tensorflow
and I got kind of hooked into their
translation tutorials at the time it was
in English to French and as you'll see
this is for specifically version 1.1
actually I don't even think this works
and 1.1 because they're using a
different version of sequence to
sequence like I don't think this
actually matches the code that's in
their github but regardless if you
wanted to run this code you really
should be using tensorflow 1.0 like you
actually can't even use the latest
version of tensorflow which is
unfortunate because this is much slower
okay anyway this is what I first found
if anybody has been following the stream
or the chat pod on Twitter
for the first while it was actually
based completely on this model so I was
like you know reading through this model
and seeing you know this one was English
to French translation but then I was
like well if he can translate from
English for French surely you could
translate English to English now as I'm
going to talk a little bit as we get
into the intricacies a chatbot is not
English to English translation it's it's
actually I would argue a chatbot is more
complex than translating English to
French but we'll talk about that a
little more later but basically it boils
down to there's no 100% translation
there's not even like three translations
that it could be there's not five
there's like millions of translations we
could have so any input in you know chat
input could have just infinite chat
outputs that would be acceptable so
anyway that's why that's it's much
harder in my opinion but regardless this
was the first model I use it actually
does work I did a I'm trying to remember
back now I think it was a 1024 by 3
model so three layers of 1024 nodes if I
recall right that was the size of the
network and that was pretty much all
that was the pretty much the only thing
you really needed to switch around
anyway this produced a decent chatbot
but I just wasn't exciting so then as
I'm following the updates coming at a
tensorflow you've got dynamic recurrent
neural networks then we've got the
attention mechanisms and the
bi-directional neural networks
bi-directional recurrent neural networks
that is and so that leads us here to the
neural machine translation that's a much
more recent and is actually still being
updated by tensorflow if you do follow
the sequence of sequence let me I just
will say it's because it's using an
outdated form of sequence of sequence
specifically because it doesn't support
the I don't know if it's officially the
dynamic RNN that's in tensorflow but
that's basically what it is it the see
old sequence the sequence didn't support
a dynamic a variable input basically so
brings us to this the neural machine
translation you can go through the
tutorial there's probably a lot of good
information here
that you could continue to learn but
basically it's it's fairly similar to
even this tutorial at least an
explaining sequence to sequence and
what's going on because basically you
know you've got the input string it
needs to be fed through some sort of
encoder and then it gets fed through the
neural network and then that output
because again it has to be converted to
numbers somehow then that gets fed
through a decoder which translates into
whatever you want so anyway this is more
along the lines of what we're going to
be using and like I said if you want you
can follow through here and you can
download it from here but we've actually
made a few changes which brings me to
the following project I've been working
on with Daniel here it's basically a set
of utilities sitting on top of tensor
flows nmt code but we did need to make
one change to nmt it might not even be
required anymore but at least at the
time nmt requires tensorflow one point
4.0 but for whatever reason in there I
forget which one it was but anyway one
of the utility scripts that basically
checks to make sure you have the right
version it checked for like a really
specific version which pretty much
always failed so anyway we made that
change and then we might have to make
changes later on so yeah you can just
grab this recursively and get that so
you can come down here and see this
lovely readme that we've been building
but basically get the package so get
clone recursive and then we'll settle
this up so I'm actually gonna go ahead
and run through that myself so I'm just
gonna pull this over here I'm gonna use
paper space you don't have to use paper
space you can use your own you can do
this locally you can actually do this
locally on your CPU you could in theory
train it on your CPU it'll be very slow
but you can so there you go so I'm gonna
go ahead and I'll put a link in the
description for a referral code if
anybody wants it it's ten dollars which
will be more than enough to get you
through training a decent chatbot which
again in production like the chat bots
on twitter that responds to you within
seconds and responds multiple people in
seconds
I made him with
or he's in production on a CPU VPS on
digitalocean
so so you can definitely run it in
production on a CPU it just takes
forever to train it
so anyways Emma on a box but I want
16:04 make sure if you are following and
you're gonna go to paper space make sure
you choose 1604 so you've got Python 3
1001 point for all that good stuff this
one's probably gonna be fine now we'll
just call this nmt tutorial
I do want a public IP if I want to
transfer some data and good creates
paper space ok so I'm gonna create that
and then I'm gonna pause it while I'm
waiting for my password and all that and
for that to get spun up and then I'll
restart as we as I can log in and
actually grab the github and all that
alright and we're back that actually
took much longer something came up but I
guess to you it doesn't seem like it
took any time
so anyways once you've got your machine
up I'm gonna go ahead and well first of
all let me pull up that github page
these Firefox nothing oops I just hit
cancel
oh it's not gonna pop up cool ok so or
go to github.com so lash Daniel I'll put
a link in the description by the way if
I forget someone remind me and I'll put
one in there Kokila
and nmt - chat bot i believe is the name
let's see if we got it right we did ok
and if we just kind of scroll down here
here are the basically the setup steps
so I'm just going to run through these
and kind of give a little bit more
explanation but it's pretty much just
this so I'm gonna go ahead and copy this
and we'll just open up terminal here
change directory into desktop and make
this a little bigger and then paste so
we're gonna clone this recursively
because we did make a slight
modification to nmt pretty small one at
the moment
but it might grow in time so you'll
probably want to clone it recursively
but feel free to take the official nmt
and see how it works for you anyway once
you've got that I think we're gonna just
change directory into here I'm the one
that wrote this this set up but anyway I
should know let me just move this over
so we'll change directory into the
chatbot directory and then yeah we're
gonna run a pip this would be a good
time to mention though you really want
to be on Python 3.6 especially if you're
on Windows Python 3.5 on Windows has
like a weird encoding bug with at least
with tensorflow but I'm pretty sure it's
all of Python 3.5 there there was a pep
I forget which one it was but it got
fixed in Python 3.6 so I highly suggest
even if it runs it starts to run in 3-5
eventually you'll hit this weird
encoding error so even though you're
opening and reading everything in utf-8
you're still going to hit it with 3/5
for some weird reason anyways we're
gonna pip but again make sure you know
your Python version and pip and all that
is Python 3.6 so pip install - art I did
not hold my requirements
text so that should just basically it's
tense flow GPU 1.4 colorama TQ DM and
reg X so there's actually a ray X
package which is a little faster than
the standard library just rake X and the
reason why we're doing that is our
tokenizer is using reg X heavily and we
want to go as fast as we can when we're
doing that for pure data on smaller data
sets and stuff it really is a big deal
but like for 2017 for example those
files for like the whole year of 2017
each of those training files is gonna be
like 12 13 gigabytes even more if you
didn't have any rules as far as like a
score and all that so anyways yeah make
sure you get the requirements and then
whenever you're done you change
directory into setup and I'm just gonna
pull it up real quick I'm not actually
gonna make any changes to setup but I
just
bring your attention to it so in setup
there are quite a few things that you
can modify here you can modify answers
to replace that's in the output you can
modify protected phrases that you
actually want to be tokenized together
so the tokenizer is going to split
things up but sometimes you don't want
to split things up I'm trying to think
it like some websites for example like
something something something calm
we we'd like to maybe not tokenize the
period in the calm we'd like you know
Google calm to be tokenized to be one
token that is Google calm not Google as
one token dot as another icon for
example so that might be a certain
regular expression that you want to
protect
then you yeah the replace on the out on
the output you can also black list
certain outputs like words stuff like
that like if you want to not you want to
make sure your your chat bot doesn't say
bad words or something like that you can
blacklist them there any way that you
can you can check those out but the main
thing that we're going to be looking at
is settings PI I'm just going to pull
that up real quick
and let's see if I can zoom with plus I
can't I'd like to make it just a little
bigger it's not like it's gonna work but
anyways you can see yeah you might have
to full screen but anyways vocab size is
15,000 here all these settings are
pretty much in place for about 4
gigabytes of vram if you have more than
4 gigabytes of vram the first thing you
should do is get vocab 2 more like a
hundred thousand with really a hundred
thousand you can get bigger than that
but a hundred thousand is kind of a good
starting point but fifteen thousand will
you'll still actually get a decent model
at fifteen thousand the other stuff I
really don't want to spend too much time
explaining all these parameters in this
video that's what I'm gonna do in the
next video
but if you want if you're familiar with
these things feel free to play around
but basically the first thing you might
want to change is vocab size but we're
not going to do anything there so just I
guess just leave that the way it is and
then we're gonna run prepare data so I'm
gonna go ahead and just do Python again
it better be Python 3 6 so my n Python
is 3 6 but make sure you're gonna regret
it if you don't do it
and then we'll just run Python prepare
data pi in this case our training files
are pretty small so I guess I should
have mentioned so this comes with some
sample data just so you can quickly run
this and test it or whatever but chances
are many of you guys that are following
along right now you probably have your
own train dot from and train dot too in
your own test files and all that if you
do before she run prepare data sorry you
should you just throw them into new data
just replace whatever is here and then
once you replace those files go ahead
and run compare data again that's
written in the RIT and the readme I just
kind of skipped over that but yeah if
you have your own data go ahead and
replace it then once you've prepared the
data again on real size files that might
take a little longer on like the eleven
gigabyte files it seems to take I'm
gonna pull it up real quick
probably about an hour ish for me to
create those files so on a smaller file
though it's not gonna take too long
anyway then what we'll do is just change
directory back to where the Train file
is and then we can actually just run
Python trained PI and then that's going
to start the network training and it's
going to output some information here
like learning rate decay factor in this
case so it's telling us a little bit
about what network is comprised with
again all that stuff like the
bi-directional stuff all that stuff I'm
going to cover I just really like the
training is what's gonna take the most
time so I figure you guys can start
training a model and then I can tell you
how to tweak it from there these outputs
happen every thousand steps so you'll
see the source is basically the input
data the reference is the real output
from your training data and then nmt is
what your chat bot responded with as you
can see this is you know damn purposes
purposes and it's just repeating itself
because it really it just it just
started so it hasn't learned anything
but over time you should you will hope
to see that this this improves the other
thing too is in as this trains inside
your model directory
you're going to have train log and this
is where you can bring up tensor board
again I think at this point I'm going to
cut it and then in the next video
I'm gonna start talking about all the
options the things that we can tweak
looking into tensor board that kind of
thing to see kind of how a model is
actually going and I can actually I can
show you a model that trained over time
and we can look at it in tensor board
and all that so that's what you guys
have to look forward to if you have
questions comments concerns whatever up
to this point feel free to leave them
below otherwise I will see you in the
next tutorial Oh
in just in case some of you are
impatient I'll just explain really
quickly you're gonna see these every
hundred steps basically it's gonna tell
you how quickly each step took it
depending on how big your model is what
kind of machine you're running it on
that might vary this is I think that's
words per second ppl is perplexity you'd
like perplexity to drop GN I think
that's gradient noise and then ideally
that's gonna probably hopefully fall a
little bit but really the big thing we
want to look at is perplexity we like
perplexity to drop and then the blue
score
we'd like blue to rise zero blue score
is horrible okay so anyway it for those
of you who are super impatient those are
the things you want to see you can start
tweaking your network to try to make
those things happen but honestly this
model especially if you're doing
bi-directional even after a first
thousand steps you should have a model
that's starting to look coherent
I'm not sure on this training if you're
using the full training data set like
the ones that we've built yeah it should
look pretty good I'm not sure on the
sample data what its gonna look like
after a thousand steps but there you go
okay so I waited I'm impatient too so so
after a thousand steps it actually still
doesn't really look that good maybe it's
cuz of the low vocab maybe it's just by
chance but actually the first one I
trained after a thousand steps it
started looking pretty good maybe that
was just by chance too but anyway
really that's all for now I'll see you
in the next tutorial