what's going on everybody welcome to
part 6 of the ML and deep learning with
the highlight 3 challenge tutorials in
this video we're just gonna continue
building on the last one which I believe
I left off fixing a slight bug that we
weren't able to train for multiple
epochs so let me quickly bring up tensor
board tensor board log dirt equals log
dirt equals lawns dang it tensor board
log dirt equals a lot slash and for some
reason I get a bunch of this I think
it's because we're kind of overriding
the steps with my crappy way of doing
things but we'll get through it so then
I found that the best way to get there
is with localhost colon 6006 if I use
like 1 27001 that doesn't work and if I
use the Machine name colon 6006 that
doesn't work either so anyways this is
what's working so let me just smooth
these quite a bit so the red line is
three whole epochs the orange and blue
is just one epoch I'm gonna switch these
to relative so you can kind of see those
compared but really after three epochs
really mmm I mean it there's a I mean
maybe it slightly comes up or at least
it levels out in the in-sample but then
add a sample not to too impressive I
have to switch it to relative to show
you the epoch that lack though again
because of the way that we're doing
things so again the orange line is
actually only one epoch the red line is
three epochs in the blue line is one
epoch the only difference is the orange
line was the time when we actually had
to load the data and prepare it whereas
these two down here we didn't so as you
can see preparing the data it takes like
three times as long so anyway accuracy
somewhere between 24 and a half and 25%
that's pretty good considering you know
random choice would be 20% and in this
case you know we've trained a model off
of you know using a threshold sure but
we're training off of model
that are moving randomly so the fact
that we got a you know four to five
percent over random is actually pretty
impressive that tells me our threshold
probably worked but now it's time to
find out so now we begin the massively
iterative process where we've trained a
model now we want to run a bunch of
games with that model and some threshold
and then the hope is that our average or
mean kind of halite collected has gone
up a little bit
any little bit will help especially in
these earlier games so so what I'd like
to do is first of all let's go ahead and
just copy training data in fact I
figured this probably that'll take too
long
what I'd like to do let me just delete
this we'll just call this training data
and I'm gonna call it underscore one
just for the first set and then we'll
make a new folder call it training data
now that's empty and then we can run
those stats that we did where we do the
histogram but also just like the mean
and all that we can run some stats and
see has the model actually improved at
all we also could do the model versus
the random and see which one wins more
often or something like that
but anyways what we shall do now is
implement the AI rather than random
incentive bot so I'm gonna copy pasted
sent a bot and now that's just sent a
bot and then it'll be - not an L will be
- M L okay let's pull that up and I am
gonna just close everything else cool
all right so we have to make a few
imports also because of the weight like
halite uses standard error and outputs
and stuff for the you know the game and
python to talk to each other so we have
to make sure we silence all that stuff
so I'm gonna make a few imports here I'm
going to import OS I'm going to import
sis we're going to say standard error
equals sista as TD air and then we're
gonna say s sista STD air equals open OS
dev null
with the intention to write and then
we're going to import tensorflow
as TF I hate that sublime does that to
me space should not or is it space oh
it's cuz you hit enter for the newline I
don't know how you to get around that
actually anyway sista standard error
equals standard error so the issue is
when you first import tensorflow
in its gonna tell you things like what
back-end you're not the back-end that
you're using but it's gonna tell you
things about like that's what scares
it's gonna tell you things about like
your GPU and and stuff like that and we
need to silence that so this is how
we're doing it basically to silence all
that output if you have it the game is
gonna air out so now sentiment ml can
remain the name that's totally fine the
next thing we need to do is continue to
silence any of the tensorflow
outputs so I guess we can still write
this out I thought about just copying
and pasting X this is just kind of
cookie cutter code that was used last
year to silence stuff from from halite
anyway OS . environ and then what we
want to say in here is TF cpp min log
level and we're gonna set that equal to
three in string and then now we want to
set the GPU options that's going to be
equal to TF dot all caps GPU capital o
options and this will be per process GPU
memory fraction and we set that to 0.05
this is a small model we don't need much
memory why do we want to do this well by
default the tensorflow is going to
allocate as much as it can and then keep
stuff just floating in memory to attempt
to make things as fast as possible for
you the problem is this is going to
exhaust a lot of power now the next
thing is I'd like to run three or four
simultaneously
simultaneous instances so you have to
keep in mind that at any given time up
to like say 16 a is could be trying to
run and make predictions at any given
time so it's really important that we
set this number very low now depending
on how many you can run at any given
time given your certain CPU and all that
you can adjust this fraction but I've
actually found it runs perfectly fine
and actually it seems to run faster on a
low GPU memory fraction and I'm unsure
why but it seems much faster like 10
times faster it's very strange
anyway I think that must be some sort of
bug or something that is very unintended
but definitely something I would look
into in the future with a larger model
even so anyways now we're gonna set our
sessions s equals TF session and then
we're doing this so that when we load in
our model it's gonna follow this session
for us and use our GPU options that we
set
so config equals TF config proto and
then GPU underscore options equals GPU
underscore options great now we're ready
to load in our models TF equals TF Kara
stopped models dot load underscore model
and then we give the model name I've
forgot the name and the other thing we
want is random underscore chance and I'm
gonna call this secrets 'don't choice
and then in here I'm gonna say zero
fifteen zero two five zero point three
five so this will be a random chance
that the AI will choose a random move
over the models choice so we still want
to have some exploration like we can't
just train models and then because the
model isn't random anymore so we want to
still have some random kind of
permutations to two for the model to
learn from new moves over time so
so anyways you know 0.15 would be you
know 15% chance that every move 0.25
would be fourth of all the moves 0.35
35% of the time it'll be a random move
and then every time we initialize this
but some boss will be 35% random some
will be 25 and some will be 15 you can
play with those numbers as you want we
just know that we want some random
exploration involved here the next thing
I'd like to do is go into models and
let's pick Oh No
oh we've made a horrible mistake that's
a huge bummer to see we can train models
pretty quickly that stinks I luckily I
think that model should be fine but let
me fix that in the training code we must
have forgotten f88 suit where is sent a
train
hmm it would've been I guess that the
model save yeah dang what I need it so
this one it will be the three epoch
model which is fine I'm gonna I'm not
sure I would have used that one the
other thing that might be wise is to
save like the score of the model so
maybe like run a model that score right
before the save something like that
whatever I'm gonna come into here and
I'm gonna rename this one models
hopefully man I just I feel pretty bad
because I wonder I hope that's a good
bottle probably what I'll do is I'll
finish the code here make sure it works
and then I'm probably gonna retrain that
model because I don't have faith that I
didn't accidentally run train and then I
stopped it or whatever after a quick
short segment so anyway I'm gonna call
this for now phase one anyway come back
up to sent a train or not sent a train
close that yeah there before we run in
trouble come back to this script come to
the top in the model we want to load
sent a bot ml we're here phase one
all right okay save the progress so far
okay now what we want to do is I think I
actually kind of want to I just want to
leave the threshold the same for now
because I want to see if the mean has
gone up like I just want to see if it's
actually gotten better or not so I'm
actually gonna leave the threshold the
same with the hope that it actually does
improve same thing with total turns I'm
gonna leave everything the same make the
same amount of ships all that and
instead what I want to do is I want to
come down to where we actually make our
our choice so choice seekers choice
range Len Direction order so what we're
gonna say is if secrets dot chase
range of int 1 out of random chance
equals 1 then we want to say choice
equals secrets dot choice will in fact
basically this
I think that's what I want so basically
if it's one out of you know 15% or
whatever as you make that range the
question is if that number is equal to
one then we'll make a random movement
otherwise what do we want to say so else
we're gonna say model underscore out
equals model dot predicts and then we
want to predict always prediction takes
a list even if it's one thing we want to
predict we need to predict the numpy
array conversion of surroundings with a
dot reshape negative one and then it's
negative one by the size by the size by
three channels so in this case we can
say lens surroundings comma len it's not
running over my face no surroundings
comma three let me zoom out a little bit
so we can see the full line model out
model that particular numpy or a
conversion dot reshape negative 1 by 30
3 by 30 3 by 3 now this returns because
you pass a list it also returns a list
so we want the zeroeth element so that's
the actual prediction now even though we
were using scalars it is still going to
output to us a a one hot vector or 100 a
so we want to say prediction equals NP
dot Arg max of model out now logging dot
info we want to save this we're going to
say F prediction and that is gonna be
Direction order prediction and then
choice choice equals prediction so then
we leave all of this other stuff and so
as we build the data the data that's
built with either our random movement or
we're not just because I'm my brain is
going slow today I don't know about you
guys let's import plus import secrets
paste this for I in range let's say a
hundred I'm gonna tab this over I'm
gonna print or actually if that equals
one print yes I apologize for this i got
I got to make sure I'm doing this right
so we're just gonna iterate over this a
hundred times if that is equal to one
and then let's say random chance equals
should be about 15 of those let's do 5
perps oh five should be about five
obviously we'd have to run just a ton
and it's random 5 and then this should
be about a 10% chance so about 10 should
be the average one we see it seems a
little low but it's probably correct
let's do point 3 so this should be a
verge of about 30 looks to be about
right okay just want to make sure that
logic was sound so in this case the
question is if this is equal to one
great
that's our 35 percent chance or whatever
okay then we might allow bah blah blah
make the choice run the game everything
else stays the same we're not changing
any of that so hot-diggity let's run it
so what happens we need to change run
game or python version no longer is it
the one that we have it's sent to bot -
ml now - L - ml and probably before I
make the mistake let's just highlight
one of these
replace all great are we ready
well I think we are ready I also think
Oh actually as we run the game we'll see
it so this won't obfuscate what comes
out so hey let's run it I forget if
pythons straight up runs well we want it
does so Python run game dot PI let's see
what happens so on an issue I shoot I
know one thing that's probably going to
cause a problem I think let me just
break this blunder what did that say
communication failed okay let's read one
of these okay that okay there's nothing
in there and then what we'll do is we'll
go into the replays I can't all check in
a second surely we're not timing out
there right no
this is gonna be a huge uh-huh if you
don't give me any error this is gonna be
a huge pain in my tuchus okay yeah okay
so we are timing out okay I feel like
I'm about sneeze I apologize fight it
okay so now no timeout so - - no timeout
and here's a little trick what I have
found is the first time the model is
loaded and the first time the model
makes a prediction it takes a little
long and then all the subsequent ones
are quite fast so for now I'm gonna I'm
gonna run this with a no timeout because
I don't care I'm trying to build data
right now I don't care about that but in
the future probably what I would suggest
you do to avoid that timeout is just
make a BS model predict before you do
game dot ready
so once you've called game dot ready
everything needs to happen within two
milliseconds so keep that in mind but
for now let me make sure I did that okay
we'll set no timeout and we should be
fine so let me just let's go again
I don't know why it's hanging okay got
it stupid error but also strange I
didn't get why he didn't error to me so
I know we're silencing errors to some
degree but then I copied sent a bot ml
and I commented out to my understanding
all of the things that we were doing
that we're silencing the output and I
just tried to run just this and see if
it would load and it would not so yeah
anyway the issue was we put we wrote
phase one in here well phase one unless
that's a directory maybe somewhere like
phase one no it just doesn't exist phase
one doesn't exist so we should have
gotten an error but we weren't getting
the error so cool anyway regardless the
AI is running we can already see here
some of the outputs I would wager I
started almost immediately whenever it
started running it looks like we're
actually getting quite a bit of halide
per per game we were already started
building some training data let me pull
up the new training data here
so immediately these I think these were
the first two games but we might have
just gotten lucky but it will be very
interesting like right now the the
distribution isn't quite so heavily
weighted towards the 40 100 s but I'll
let it run for quite a while I'll
actually open up a couple more run quite
a bit I'd like to get at least another
you know you want at least probably
somewhere between five and ten thousand
games if you want to make the fair
assessment but these take so much longer
to create so we'll see if I'll actually
have the patients do that but I might
let it like run all weekend or something
but the real question is is it actually
better than the previous one but as you
can see here all of these games saved
because they were all great games so
anyways once I have maybe about a
thousand games or something I'll check
the distribution and see and if it's
pretty darn good we can figure out what
threshold do
you want to use and so on so anyways
that's it it was a pathing issue stupid
on my behalf but also I'm curious why it
didn't why that didn't come up even when
I created the copy script ran the copy
script and still like why didn't I get
an error that said hey this doesn't
exist very curious but whatever that's
it for now questions comments concerns
whatever feel free to leave them below
otherwise I'm gonna see you guys in the
next video where we just basically we'll
check the distribution here and then
continue that iterative process and just
kind of I'll probably I don't know if
what I'll do in the next video if I'll
maybe already do the iterative process I
don't know or maybe the next video will
talk about the changes and then from
there do the iterative process because
basically after like at this point
nothing new happens really at some point
we want to add more ships and we want to
change the threshold and all that yeah
this clip in my opinion this AI is way
better
I mean it's he is I mean we don't have a
5000 game yet but look how few of them
are at the 4100 s almost instantly we're
in the 42 s and up so I would hazard to
guess this is a significantly improved
but we'll see just by how much in a
while anyways that's all for now see in
the next video