what's going on already and welcome to
part 9 of our AI in starcraft 2 with
python tutorial series in this video
we're gonna continue working on what
we've been working on which is applying
deep learning to Starcraft 2 now of
course before we can do any machine
learning we have to build our dataset
and that's basically what we've been
focusing on so far so up to this point
we've got quite a good visualizations
quite a good visualization of the map
and what's going on I just want to add a
few more visualizations and that's gonna
be things like how many minerals do we
have how much gas do we have and then I
also want to see a visualization of our
like our just our population supply how
much of that I've we exhausted like a
ratio of that and then also like a ratio
of our military units to our worker
units so we can see on the map usually
pretty well like how many military units
we have versus how many workers but
things that we can't quite as easily see
is like how many units we have compared
to our supply I mean maybe the network
could learn to count the pylons or
something and figure it out it's totally
possible but I thought it'd be somewhat
useful to display certain things in like
ratio form so that's the last thing I'm
just going to add and and this
methodology of displaying things isn't
necessarily gonna be learn a bowl by the
network I think the a bar length is
totally doable like how long the bar is
but in general neural network artificial
neural networks tend to learn more like
patterns rather than like a length of a
thing but I'm just gonna throw it in
there anyways so just to see so anyways
let me think I'll just do this again
there's not too much point this should
be self-explanatory type of code let me
go to where we're drawing things and
just after neath after neath underneath
herbs we will just paste all this boom
so what we're gonna do is display our
minerals so the game really would have
no idea of knowing like how many
minerals do we actually have so that's
how I'm going to show it just with a
little ratio of a bar if we have more
than 1500 then we're just going to show
it as like a full bar and then we're
gonna do the same thing with the vespene
we're gonna do something similar with
population in this case we're going to
say supply left over supply cap so how
close are we currently do our cap again
visually the network could maybe learn
that but given if units were over it you
know maybe not so I want to make sure we
have that totally clear and then and
then I have like a military units kind
of ratio compared to our workers and
then actually in this case this is just
military units over our entire our
entire supply and I actually don't see
plausible supply oh I do see plausible
supply okay my medicine okay cuz 200
could be the max anyways let's try to
see if I ever actually use that okay so
anyway then we come down here and then
we just draw each each line then we're
just kind of color coding those lines a
little differently so let me just go
ahead and run it I think that the
visualization of what's going on will
just help quite a bit as we have done
recently the recent members slash
sponsors on D hannes Kalama with Sperry
Oh Howard Liang and lizards Mona thank
you very much for your support you guys
are amazing okay so as you can see here
we have the you know the bars basically
and these are in order so I actually I
purposely flipped them around so this
one here is worker to supply ratio and
then we have the plausible supply so
this is just our current supply out of
200 so as this gets bigger supply oh
yeah
cuz we just built the nexus and then it
bumped up so actually supply is well
yeah okay that's our that's our
our quota okay I'm blanking on a good
word anyway and then the alloc down here
this is how much vespene we have and
let's just check to make sure we've got
a lot of Espeon and not so many minerals
yeah so we're definitely we've got quite
a bit and that's usually the case we
spend minerals much quicker or maybe
we're getting best mean much quicker but
yeah okay so so now that we've got that
visualization the the I think the
visualization is pretty much done there
are some things I'd like to do like when
the Scout flies you can see how like as
it goes up it loses units and as it goes
down it loses units again I'm not really
sure if we can get that radius a little
larger because I do like how he like
flies around randomly I don't think I'd
want to hold him in the same spot
another option you have is to just keep
sending Scouts honestly I just thought
of that just now that's probably a
really a better more efficient where the
heck is it this one yeah how many spaces
it so only takes in one Oh takes away
one of our spaces and I believe
generally we actually pretty much always
have enough supply I don't think
supplies ever an issue until really late
game I wonder about sending like an
observer here here you can't see what
I'm pointing to get back up here sending
like an obscene you can't see this one
unfortunately but but like up you know
up to the did you can see this mouse
okay so an observer here here and here
like these are gonna be like if you
actually look at these maps the reason I
like your your start point here is it
has basically a choke point there's like
only one way to get here and you've got
to come through that choke point and
then as you expand out like here and
here there's like two points but then if
you go any further than that it's like
you can get attacked really easily so
it'd be useful to at least send three
observers out at any given time to each
of these locations and just always hang
one over there I actually think that's a
really good idea I think
to do that unfortunately I've already
begun training data so I'm gonna go
ahead and see this one through but I
think in that if this is anything
promising at all I might want to change
the data to something more like that
that would be much more informative I
think all right so that looks good I'm
happy with the little bars down here and
maybe a little bit useful and maybe they
won't who knows we'll see but we won't
know anyway okay so now the last things
that we need to do are modify the attack
method I think that's really it modify
the attack method and then we actually
kinda need to add one more one more
thing so yeah so let me so first of all
yeah I guess I will do the attack method
first and then I'll talk about the other
changes that I've made so coming down
here I'm just gonna replace our attack
method with the new method boom exit out
of that so what we want to do is we're
just gonna choose a random choice
between zero and three pythons random
isn't the most ideal so you'll notice
there'll be a lot of patterns and stuff
like that but it should still be over a
long enough time frame produce at least
balanced data now we're gonna say target
false for now but if we do find a target
will attack that target and the way
we're going to find that target is if
choice is zero we're just going to wait
a random 22 165 and then we're gonna
modify this self ourself that do
something after and we're just gonna say
that it's the current iteration plus the
weight so self don't do something after
I don't think we've defined it here
we'll add that to our init method here
in a little bit but we got one more
thing we're gonna have to add so I'll
get there so so if the choice is zero it
just means we're not we don't want to do
any attack we want to wait so this would
be a scenario where you know ideally the
AI would learn that when I don't have
many units to send out to attack
it's a bad idea to attack in general if
you just send like one unit at a time
you're gonna lose it's not gonna end
well for you so again just through a
simple like of an evolutionary process
we should find that that's a
unsuccessful thing to do so we should
have training data that over time
doesn't do you know it waits to build an
army that's of a size that makes sense
for the enemy that we're gonna fight but
also you know you might need a few less
to defend like your little base but you
might want to have like I don't know 10
at least void raised before you go to
the enemy base but if you're just trying
to defend yourself against a smaller
little army that they just sent to you
just to give you a hard time or whatever
you might not need enough as many but
also you might not always want to
address when they send like a little
fight over to your base you might want
to ignore that and just go straight to
their base and decimate them so just
keep that in mind as well especially if
they send their entire army if you can
defeat their like all their buildings
before they can defeat all your
buildings you'd be better off just going
straight to their base so anyways if the
choice is one we're just saying okay
that's going to be attack the closest or
attack the enemy unit that's closest to
one of our nexuses and so we can just do
that with the following little string of
methods here so it's just self dot known
enemy units and then closest to is an
attribute that's just built in and then
we're just saying closest to a random
choice of one of our nexuses you could
iterate through all the nexuses and then
figure out which enemy is closer or
whatever and that would probably be a
way better idea because maybe they're
not close to any of the nexuses but if
that's the case like if they're close to
really close to one but not the other
ones you've probably expanded really far
in the map and you're really late in the
game and it's not a big deal at this
point like you've made a mistake if you
made it that far probably okay choice
two is attack enemy structures super
simple just random choice of known enemy
structures interestingly choice I wonder
if I haven't actually seen that error
out but I don't think you could choose a
random choice of none
I feel like there should be a try and
accept in like all of these do I not
have I can't believe I don't have that I
haven't seen myself hitting a lot of
errors but I can only imagine I guess by
the time you've hit like you can start
iterating through these like the only
time we will ever go through these is if
we have vital I don't voice raised so I
guess maybe that's why it's been so rare
but there's going to be times where you
you can't make a choice because you
don't know of any enemy structures so
that could be pretty problematic or that
could be problematic like late in the
game like if there's an enemy structure
kind of hidden away and you just never
saw it should probably try and accept
that one but I'm gonna leave it for now
and then finally three attack the
enemies start if we do have a target
will attack it otherwise forget about it
and then here we're just making the
training data again just like many of
our examples in deep learning
you know if the choice is zero that
means like the 0th element is one so
we're using a one hot array so we want
the neural network to produce an output
like this and this would be zero you
know it would be no attack conversely if
you had something that was like zeros
zero one zero that would be 0 1 2 so
attack enemy structures okay so that's
all that is alright so now we've got the
data but then you'll see we're appending
the training data to this self doubt
train data so that's the other thing we
want to go ahead and add to and we
probably don't need to print this out
anymore that was just for debugging
purposes so then I'm just going to go to
the top here and inside the init method
I am going to add a couple of things so
first we're gonna say self dot do
something after I'm going to say that's
equal to 0 and then self dot train
underscore data for now will be an empty
list and then at the end of it we're
gonna save it but we're gonna talk about
that in just a minute
the other thing I kind of
talk about do something after I feel
like maybe I didn't stress that one
enough so why why would you want to wait
let's say the choice is zero and it's
don't attack why am i waiting there for
iterations well I don't know if this
range is the best but you you want to
wait because because every iteration
we're gonna come right back to this
question right which one random choice
do we want to do so if you don't force
them to wait for some time when the
choice is wait your training data is
going this option will always be a
worthless option because like literally
in the next iteration your AI randomly
chose to do something else but in
reality if do nothing is really the good
choice then you actually probably are
gonna want to sit there for more than
one iteration you'd want to sit that
you're probably like if that's the good
choice you should sit there for who
knows how long but some duration so
that's why I want to wait there because
otherwise the weight would be worthless
it wouldn't actually you know you'd have
a such a small chance that you would
wait for more than even one or two or
three iterations so I wanted to just do
a random range of 2165 and those numbers
I just made up so so yeah cuz like think
about it this way if choice is one what
happens well that unit is no longer idle
and it's probably not going to be idle
ever again it's going to die busy so so
so that's why we want to make sure that
these think these choices really are
balanced choices if anything you might
want to wait a little longer than that
but 165 is a minute in my times there
might be a way to like reference game
time or you could use the time you know
key well you can't really import time
unless you play in real time I don't
think I've seen a game time if someone
sees a way in there might be something
in the body a class to reference the
game time someone knows how to do it
post it below cuz I don't think I
haven't seen that haven't come across it
but it would be a better choice than
using iterations
okay so okay so that is our attack
options did we add everything we wanted
to use of train data do so okay I just
wanted to explain to you something after
okay so the final thing is we want to we
want to be able to reference the result
that we get from the game because we
want to save the game if the result is
we won but eventually we're going to
want to be able to Pitt sent a bot
against itself so in order to do that
and save the result I feel like the best
way it would be to use some sort of
built-in method that knows who the
winner is unfortunately when the game
ends your bot doesn't know it so that's
not very good so in at least for the
time being the the choice that we've
gone with is to add in on end method to
the bots so Daniel wrote basically
forked the the Python SC to package and
then added in on end method so all you
really need is one take that package and
all you need to do is like I'll put a
link in the text based version this
tutorial and what you all you need to do
is just download and unzip the package
put that in your you know your project
directory and it will use that there
other than that you probably don't need
this this is where I'm saving the
training data on the server but maybe
like here or maybe local here this might
be your on end method and you don't need
to you don't need to call this yourself
if you use the updated package the on
end method is called now other than that
I'm trying to think if there's anything
else I need to change
there kind of is I probably need to pull
that package because I can't I gotta
test it and make sure it works so let me
copy I think that's actually this copy
paste yeah okay so hopefully in the
tutorial code now I have the SE -
updated package so so if you download it
that it should just be this and you'll
have all this stuff you might I forget
if what she's got in there I don't know
if he modified this for himself or not I
forget oh I modified it I wonder if he I
can't remember anyway doesn't matter so
if you wanted to by the way you can
import OS and then use something like
this so if your path to Starcraft is
Starcraft stuff and then Starcraft 2 you
can change your path that way and it
will find the maps for you if you want
to change it but I've modified mine okay
so now look I think all I would need is
import time and that should be about it
the only other thing is since I wanted
to run this headless to create the data
I also did add a headless equals true
and then down here I just basically did
this like if if not or rather if yeah if
not headless I can't think straight
anymore
boom should be what I did there yeah so
if you if you wanted to visualize data
you could but I've been running this on
a an actual Linux server so we don't
really want to see it anymore
what you couldn't it would throw an
error if you tried to show it which
reminds me when we add stuff to train
data just take note we're adding self
dot flipped again don't screw up and add
the resize that would be a waste of data
okay I believe I believe we're ready to
run
let's see hopefully I don't hit in there
Oh
or did I I think I said it to true huh I
did so we won't visualize anything we
should see the game though this debt
that won't apply to the pop-up game I
just want to make sure it actually runs
and then if it does well it's probably
gonna yell at me because the directory
that I asked it to save in doesn't exist
so on end train data I should probably
add that in so I'll just do this new
train data okay hopefully it'll save it
in there but also it will only save it
if the result is a victory which it
probably will not be because we're
fighting difficulty hard so maybe I'll
change that to easy or were we
victorious has no attribute flipped self
dot oh wait hold on okay so right so the
other thing that this the results of
copypasta so it should just be you
should just need to say self dot flipped
boom and then when you add when you do
the resizing self-doubt flipped because
we want to be able to access this
variable elsewhere so yeah that should
be it all so now we're fighting easy and
headless I'm gonna change this to false
for the next time if there is another
error now regardless of however this
pans out you do not have to make your
own train data I am going to do it this
will take forever to make good data
because you probably don't really want
to always fight the easy bots you
probably want to just fight the hard one
to create good quality data so that's
taking a really really long time so I'm
running about a hundred games
simultaneously just so I can collect
data head
reasonable rate and I'll just share that
data with you guys so you don't really
need to worry about it but if you wanted
to like modify things and run it
yourself at least now you have a good
idea of how I you know went about
collecting the data and all of that so
is this our observer yeah so we won't I
won't really know until we actually have
void rays yeah like right now it's just
start a little observer but that should
work should not throw an error
okay so now we do have a unit can't find
drop-off I guess just for a probe and I
can't recall I think we're fighting the
easy easy bot so we should actually come
through with a win it should save the
result and then all I did was just save
it under the time so like whatever the
current integer value of the time is is
what I saved it as where is sublime
there we go so on that on end method I'm
just saving it as wherever the time is
basically all right game let's hurry up
and kill this guy so we can make sure
our data saves it's not looking very
well for him if anybody has a better
idea for how we can let's see f2 there
we go
for how we can if there's a better way
to access the games result I I know you
could say like results equals and then
the run game and get the result that way
but then if you ran to bots I'm just not
really sure how you would go about about
doing that unless you like random in a
really specific order maybe and then you
could like just know the order so I'm
not really sure but this is the path
that we chose this game better be over
soon there we go so on and cold we got a
victory and then we come in here and
boom there's our training data and the
training data isn't early all that long
is usually like 30 to 60 choices max per
game that's why you want to run a lot of
games because they take a while okay so
that is it for now the next time you see
me I will have a gigantic training set
for you well a sizable training set
that's good enough for training neural
networks anyways if you have questions
comments concerns suggestions have I
made some mistakes some stupid choices
let me know below I know you will so
anyways
that is all for now I will see you in
the next tutorial