what's going on everybody and welcome to
part three of our deep learning and
halite and part 6 of our just overall
highlight tutorial series
in this tutorial we're meeting is
training a model on the the training
data basically that we've been building
I'm I'm just gonna keep letting this run
in the background actually it's like
this plus 40 I stopped it then I was
like I probably should just keep it
running while I write the training code
so we'll just keep it going so I almost
actually have a hundred will definitely
be over a hundred bytes how we're done
so with that let's go ahead and get
started I'm gonna call this model
trainer so I'm going to copy data
creator dot PI call it model trainer dot
PI edit it and fit that on our screen
alright so what we're gonna need to
basically what you're going to need to
have here is gonna be both tensorflow
and caris you could just do pip install
tensorflow and pip install Kerris in
your command line but tensorflow
you probably really want to have the GPU
version so it should be tensorflow - GPU
and in order to do that it can be cut
tedious to install so if you need help
installing the GPU version of tensorflow
on both windows or linux go to the
text-based version this tutorial i've
linked to both options in that text base
tutorial it'll be in the there's a link
in the description to the text-based
tutorials so head there if you need any
help I'm also using TQ DM so pip install
Tiki DM you you don't need this I'd like
to use T qdm anytime I've got a large
for loop that I'm curious how much
longer it might be but by no means you
do you need it so anyway but grab if you
want so to begin I am going to I think
I'm just gonna keep doing what I was
doing before I think that I think it's
just more useful we've got so much data
and so much code to cover to do it this
way so let me just I'm just gonna copy
this over so important Karis cuz we're
gonna use it
sequential that's just for your
multi-layer perceptron feed-forward type
model we want dense layers we want
dropout which drop out activation at
that point also if you want to learn
more about deep learning in both Python
and just deep learning in general I've
also got links to tutorials there as
well if you need more information on
that so anyway that's that load model so
we can save and load models if we want
random for shuffling our data TQ DM like
I said it's for iterating over models
I'm sorry
in the it's for producing like progress
bars basically for anything you're going
to iterate over and then numpy for
obvious reasons so continuing along
we're gonna have a batch size this is
just how much how much of a batch are we
gonna feed through the neural network at
a time I'm gonna go with 128 I found the
128 to be like the sweet spot so so so
I'm gonna go with that epic there epoch
says everybody likes to correct me will
go with 10 somewhere between like 1 and
10
it just kind of depends I wouldn't I
haven't really found any benefit to
going over 10 but you can if you want
feel free to play without all this stuff
and then test size how much of our data
do we want to dedicate to add a sample
testing now I'm gonna go ahead and do is
create in bottle and out model again I'm
gonna copy and paste this one's gonna
take a little bit to write I'd rather
just explain it basically this is going
to tell us what is our batch size and
how many pox did we do so it's going to
save these models now we don't have an
input model to start with so I'm just
passing 0 0 there but if you had one
that you wanted to caillat you know you
might you're gonna find that there's
times where you train a model and then
maybe you get some more data or you get
some new data and either you want to
train on our larger data set or you want
to like you want to combine your new
data with your old data train a couple
epochs there or maybe you've got pure
new data that you just kind of want to
throw at it there's all sorts of reasons
why you'd want to input a saved model
but anyway
now we're gonna do is if we did have a
we'll load it so I'm gonna say load
previous model will be false but if we
want to load a model then we'll load in
that model what's really cool about
Kharis I haven't actually covered Karros
on my channel so if you're on my channel
and you haven't used Karis yet one
really nice thing about Karis /
tensorflow in general as well as TF
learn which is what I usually use in
place of Karis is you don't if you load
a model you don't have to define the
model which is really nice I always
thought that was really dumb that you
had to define the model and then load it
if you saved the model it just seemed
like cut why can't we just save the
model maybe that's something new with
tensorflow like 1.4 or something like
that that I just haven't experienced yet
or something but at least Karis does
that and I think that's great so
continuing along now let's go ahead and
read in the data so again I'm just gonna
copy and paste this so here we're just
opening the train file we split by a new
line and then we're gonna Val this stuff
so one option we could have done rather
than appending to the file is like save
it as some sort of numpy file or
something like that but when we're just
saving it as this file it's the format
is a string so we need to use eval I
because it needs to be evaluated for
what it is not be a string so this
basically converts it to actual data not
string anyway that's our input that's
our output now we need to discuss
balancing so with with all machine
learning but deep learning too
you want to have balanced data so in our
case we have three choices we have
attack mine our own planets mine and
empty planet
chances are aside from the initial part
of the game chances are the most common
actual output vector is attack because
most of the game you're going to wind up
with all the plants or ocupado and now
you just need to attack and so it's
probably gonna be the case that like 75%
of our data is attack attack attack
attack and the problem is those two two
major issues one it's hard to assess how
act
here you are if you have unbalanced data
but it's also hard to train a model with
unbalanced data because the model is
gonna learn like let's see you know I
can be 75 percent accurate if I always
say attack because that's what the model
is going to try to do it's gonna try to
figure out what's the quickest way to
get to the best answer and so it's kind
of just if 75% of your data let's say is
attack it's gonna very quickly learn
okay I just should always attack I
should I'll be the most accurate so we
want to make sure our data is perfectly
balanced for that reason but also when
it comes time to evaluate our model and
how accurate the model actually is if
your data isn't balanced it's really
hard to determine are we this accurate
because we're always predicting attacker
we're most often predicting attack
because the model incorrectly just
learned to just in general predict
attack or is it actually smart so it's
really hard to determine how you've
actually done if you don't have balanced
data so there's two reasons why we want
to do it so what I'm gonna do now is
just put in this code for the balancing
so we have these lists that are attacked
mine in mine empty planet there's got to
be a better way to do it than this but
this is the way I've done it so so
basically what we're going to do is if
the output layer is attack we add it to
attack if it's you know mine our own
planets we add it to mine our own
planets and if it is mine an empty
planet we append there and then what I
want to go ahead and do is like for now
what we can do is we can just print this
we can just print it out so we can see
the length see what we're starting with
and then what we want to do is grab
whatever the shortest one is so we want
to know what is the shortest length
thing I'm gonna wager it's probably mine
empty planet but that's okay
then what we want to do is we're gonna
shuffle all of those lists and might
throw it so dry and so you want to
shuffle them for a variety of reasons
but mostly we just chances are if we
don't like at least shuffling at this
stage isn't as important as shuffling in
the next stage but in general you just
don't want data necessarily to be in
succession unless you want your data to
be in success like if we were doing like
a
a recurrent neural network or something
it would be great if the data was in
succession but we don't actually really
want it to be in succession here so
we're gonna go ahead and shuffle that
and then now we're gonna make all of the
data the length of whatever the shortest
is no if I've got al abuse today or what
my eyes are itchy my nose is itchy
throat is dry okay so now once we've
done that let's print the new lengths so
now we get the lengths these should all
be identical in length now and now we
want to add these all up together so
we're going to say now all choices are
just attack plus mine plus mine our
planets plus mine empty planets and then
we want to shuffle them so here it's
super important that we shuffle
otherwise the model as we feed it data
the Mont and probably the testing data
for example would only be like mine
empty planet always but then also for
here it would be really hard for the
model because like the first large sum
of data would be all attack attack
attack so the model would very quickly
learn ok attack and then it would shift
over to this chunk of data it'll be all
mine our planet mine our planet and then
it'd be like oh shoot and it would shift
over to always mine our planet and then
it would get to my name she planet and
so on and that wouldn't be ideal for
learning so we won't definitely want to
shuffle them and random shovels in place
so this should be fine okay now once
we've shuffled that data we need to
parse it back to train in and train out
information so we're just gonna iterate
over and again I think I use it up here
too so up here I use T qdm so from T qdm
import TQ DM and then I just use TQ DM
over this enumerate train in if you
don't have to you could just remove the
TQ DM encasing there and you'd be fine
if you don't wanna use Tiki TM I just
really like TQ DM so anyway yeah so now
we've appended it and then so we don't
have to do this every time like if you
haven't made new data you don't need to
do this every time so we're gonna go
ahead and save it to train in and train
out and then to load it back in you
would just do NP tot load training train
out and then what we're going to do is
what the data to training and testing
boom and then we build the model so
because it scare us and it's awesome if
not load previous model then we need to
build the model and define it but again
if you have that model and you load it
you don't have to do this it's great
it's just great so in this case we've
got a 2 by 256 hidden layer model 50%
dropout and that's about it
so you can you can feel free to tweak a
lot of this stuff on this I'm just
throwing a bunch of information at you
there's so many variables in both the
starting script that does the random
stuff there's so many things that we
could change there and tweak to make
things better and then obviously in the
neural network model there's a bunch of
things we could tweak you could tweak
the layers layer size you could tweak
drop out how many layers we have you
could treat change the activation layer
here you could change I probably would
never change that activation layer but
you can feel free to change these
activation layers and all that and again
just like before if you if you don't
know much about deep learning I highly
encourage it that you you can go to
Python program net come either come over
here if it's small or there should be a
search bar right at the top and you can
probably just type in deep learning boom
you could definitely check out this this
but you can't see it this this series
here which is the full machine learning
series which covers basically all the
machine learning classifiers well not
all but a large amount not just deep
learning and basically that was kind of
structured with the idea of we go over
the classifier show like a practical
example and then we code that classifier
ourselves from scratch and it just at
least for me but I think for a lot of
people that helps to really solidify how
this stuff works
but if you want to start with just deep
learning you could just go here the
introduction to deep learning with
tensorflow
and yeah you can go through this if you
if if you want to learn more about it if
you already know about it
good for you so back to what I was doing
now I left my page on amateur yeah once
we've built the model we're ready to fit
the model now this would be outside of
that load previous model because we
actually
train so to fit you pass the training
data batch size how many epochs you want
to do verbose that has to do a tensor
board I'm not going to get into tensor
board here but if you want to learn more
about tensor board go to the deep
learning series and then the validation
split this is gonna be a 10% split so as
its training it's going to output some
accuracy for you but that accuracy is
going to be based on a portion of the
training data so in theory that's in
sample accuracy and you know it you take
that with Grant salt so what we really
want to know is the out-of-sample
accuracy so to generate that we're going
to say score equals model that evaluate
on the testing data now it's a good idea
to keep this as a value of something
because of this validation accuracy does
not line up to the score so the the out
of so if the in sample accuracy is much
higher than the out-of-sample accuracy
chances are you have over fit so it's
good to actually track both because it's
a good indicator of when you've done too
many epochs but yeah
so we'll keep that around and then we'll
go ahead and save the model just case we
want to look later says where it saves
to tell us the test score tell us the
test accuracy so so this is gonna return
both the score and an accuracy for the
life of me I can't remember with this I
want to say the score is like son I'm
not even I'm you'll have to look up what
score is well maybe I'll look it up real
quick I can't it has like some sort of
special meaning to it but I just I
forget what it was
Karis score I think it's to Karis test
score I can't remember what it means
because one of them's it's not it's not
like a percentage accuracy so score is
the evaluation of the loss function for
a given input I'll let you do more
research if you want I've never done any
use of that so I've never really paid
much attention to that okay so at this
point I've got about 200 games done so
I'm gonna save this I'm gonna break the
trainer so yeah as you can see hopefully
these are about 50/50 yeah pretty close
okay and now we've got the the the model
trainer let's go ahead and see if we can
actually train a model so python model
trainer pi oh this might be 3-6 I'm not
sure if I've got TQ DM on three I do
apparently so this is YT q DM is
wonderful look at this beautiful loading
bar how could you not love it so we have
a hundred and ninety two thousand
samples but once we balance it should
change quite a bit okay so yeah as you
can see attack is a hundred and
sixty-six thousand and then the other
two is twelve thousand and thirteen
thousand so then once we balance it all
three or twelve thousand two hundred
fifty three and then coming down we go
ahead and run as we can see we got an
accuracy of fifty seven point nine one
at the very end and then the test
accuracy was actually higher than that
at sixty-one percent so that's good if
the test out if the out-of-sample is
better than the in-sample you're doing
good okay so with that first let's talk
about this really quick so forty-five
percent accuracy on the first epoch
what's random so we have three choices
random would be thirty three point three
three repeating so actually even out to
the first epoch we're better than random
but but by the last one we're almost at
you know two thirds which is pretty darn
good for three choices obviously we'd
like to the more the merrier but that's
actually not that bad so anyway now
a model that at least for the winner
input and outputs we're about 61 percent
accurate especially considering how
similar mine empty planet verse mine our
own planet is not bad we probably should
have gone more at least in this case
with a binary choice another good choice
to make would be a flee mechanic so you
know attack mine and flee are probably
three better three choices so by flee I
mean run from the enemies so a really
good mechanic to waste your enemies time
and resources basically because a ship
itself is resources because that ship
could either be attacking and depleting
some other enemy or it could be mining
so it's really important like if you've
got two ships chasing one ship like two
enemies chasing you or one ship or more
it's in your best interest to flee and
waste that resource right so anyway just
an idea so now what we're going to do is
take this model how do we take this
model and and run it on halite so that's
what we're gonna do in the next tutorial
is what do we need to do to upload this
to halite there's a few changes and
tweaks we need to make to the script in
terms of using the model and then
there's also some tiny things that we
need to do the other thing too is we
should probably run it locally just to
see if it works but anyway that's what
we're gonna do in the next tutorial if
you've got questions comments concerns
whatever feel free to leave them below
otherwise I'll see you in the next
tutorial