what is going on everybody and welcome
to part 7 of the AI in Starcraft 2 with
Python series in this video in the
coming videos we're going to be getting
into what I know everybody has been
waiting for and that is deep learning so
up to this point we've got just a simple
rule based AI even though it's just if
statements some people have seem to
confuse it is still AI but what we're
going to be doing now is is implementing
deep learning now right out of the gate
I just want to say first of all this is
a completely unsolved problem no one has
solved you know Starcraft tuned by
solved I mean you know defeated expert
players let's say no one has done that
they have done it in more simplified
games like you know if we can remove a
lot of complexity from the game we can
definitely defeat human players already
but so far the actual game itself and in
its entirety you know allowing let's say
eat uh step to build all kinds of
different units build different
buildings research different upgrades
and stuff like that that has not been
defeated because it's just so
complicated so so first of all let's get
that out of the way second of all I do
not know the answer to this problem all
this is going to be at this point
forward is my attempt at solving
Starcraft 2 is deep learning alright so
how are we going to do it so the first
thing is especially with an unsolved
task like this is there's there is no
tutorial that says oh use this network
or whatever we have to figure out how
we're going to solve it so right out of
the gate when I have a problem like this
the first thing we got to think about is
what you know what do we hope to achieve
but then how can we do it so we know we
want to use deep learning but first we
need to just like dramatically simple
simplify what's going on here and at
least for me I think the weakest point
in our BOTS at the moment and a
relatively easy task would be the attack
method here so
I can simplify the attack to be you know
don't attack defend our base attack the
enemy base or attack enemy buildings
attack enemy units so I can break that
down into four really simple choices all
right and then we could take those
choices and we can either apply them
unit you know by unit or by entire army
or entire idle army so entire army
that's not already performing some
objective so I think the best starting
point would be to apply it to the entire
idle army then maybe later apply it you
know on a per unit basis I think that
would result in really cool things but I
think it will overly complicate things
to start okay so we got that out of the
way we know what we want the output of
our neural network to be now what's the
input what's the structure of our neural
network well I know a lot of people are
gonna be like a lot reinforcement
learning and the only reinforcement
learning everyone seems to know about is
cue learning I'm not gonna do cute
learning here
I think cue learning is way too going to
be way too complicated for this kind of
a challenge where you know you take a
path and then if you got the reward you
you know divvy up that reward amongst
all the the steps you took along the way
in your path
first of all it's going to result in a
huge amount of training data like but
but not really meaningful it's just
gonna be super noisy but also it's just
it's not really the best solution
I don't think here so other than being
clickbait tea I don't think cue learning
is actually how we're gonna solve this
again I don't have the answers just
don't think it's going to be that there
are other forms of reinforcement
learning surprise but I think what I
want to go with is more like an
evolutionary learning method I think
that's most fitting here too so the idea
is to have an AI that makes choices and
if it makes good choices and it wins the
war well congratulations you get to move
on your you become a continued asset to
the gene pool
and if you lose the battle well you're
forgotten to history you don't get to
reproduce into the future so I think
that makes a little more sense here and
especially if we are able to get down to
a you know a unit by unit or something
like that that could be kind of cool but
really the the victorious algorithm
either wins or loses as a whole so
anyways that's the way I want to do it
okay so that's kind of the structure of
how I want to do things we still have to
talk about okay what is the neural
network that we're going to use because
basically what kind of input do we want
because basically when you choose a
neural network you gotta think about
what kind of input we're gonna put in so
again we're just don't we're going from
the end forward here starting with you
know what did we want our neural network
to output it what are we going to you
know what are our hopes for in terms of
how it's going to actually learn and
yeah basically learn now what are we
going to put into this neural network
and that's going to basically force our
hand was what kind of neural net we're
going to use and the issue here with
Starcraft 2 is there's so many variables
that are you know there's almost a
variable number that well there is a
variable number of variables and even
our variables have variables so you know
each unit is a variable but then the
type of unit and where is that unit all
those things come into play that's just
very complex it's very hard to feed it
into a neural network so immediately
we're looking at some pretty complex
networks so to me I think the simplest
network that you can pass any amount of
data and any variable data to would be a
convolutional neural network because you
could visualize everything so first of
all you can visualize the unit you can
color-code units based on what they are
you can size them you can also visualize
things like our our materials like how
much minerals do we have in the gas that
we have you can you know visualize that
with a bar or something like that and
the confidence should be able to figure
that out so that's my plan going in and
the first thing that we have to do is
basically well there's two things we
have to kind of modify this a little bit
we want to further simplify because the
you just guys make things as simple as
possible to start if you're successful
make it a little more complex so the
first thing I want to do is I just I
don't want the stalkers anymore we're
gonna go with void rays later on we're
probably gonna need stalkers because
right if we only do void rays we really
can't defend ourselves against early
rushes so just keep that in mind
otherwise I think I'm gonna leave the
attack method the same because we're not
going to build stalkers anymore we don't
we only need one gateway and we only
need that gateway to build the
cybernetics core which we need for the
Stargate so so rather than this logic
I'm just gonna get rid of this and make
that one so if we have less than one
gateway well let's build one otherwise
who cares and then we don't want to do a
half we'd like to just build well let's
just get rid of the divided by two and
then these extra parentheses here all
right so let's see and then build
offensive force we only want to build
the void Ray's okay so that's the
simplified logic so now for our attack
we only have one type of unit and all
that we again we can make that more
complex I don't see any reason when we
use the confident that we can't color
code and size the stalkers a little
different I don't see any problem with
that it's just right now I want to keep
things as simple as possible if I find
this to be successful we can try to add
other variables and see if that you know
is acceptable or nope that throws the
whole thing off and it can't figure it
out or whatever so the first thing we
always want to do in a case like this is
just let's make this as easy as possible
to learn for a neural network so now
that we've got that the next thing that
we need to do is one modify the attack
method to attack based on some sort of
rules to start there just going to be
random and then we're gonna let random
you know with just run a ton of games
and any time random wins well we're
going to take the set of actions that
random took and we're going to say hey
these were
these were good actions good job army
and we're gonna use that as training
data any of the armies that lose
randomly we're gonna use that won't be
training data now if what we're gonna do
is against the easy bot you could just
win randomly for no good reason against
the hard bot I'm pretty confident if you
do things smart or someone intelligently
you'll win and if you don't you'll lose
so my guess is this will result in good
training data again I don't know I have
no idea so so come along okay so so
we're gonna need to modify that but
first let's do the let's do the input so
what is the imagery that we're gonna see
so to begin we're gonna use open CV just
for visualizations sake you you really
don't actually well yeah you should have
opened yeah you're gonna need it because
we're gonna use open Stevie's drawing
methods and stuff like that so we're
gonna import a CV to and import numpy if
you don't have these numpy should be
relatively simple to install it's very
much like all your other packages CV 2
is not if you're on Windows if you are
on Windows head on over to Python
permanent net and do a search for open C
V okay and hopefully you'll you'll find
in that first tutorial how to install it
on windows and all as well as like Linux
and stuff like that it's a little easier
on Linux windows you just need to go to
that unofficial sources okay so once we
have that we're gonna create a new
method and I'm gonna call this new
method Intel so the Intel method
I'll just define it or make it right
here a sink let me zoom in a bit
define Intel self so Intel will just be
the visualization of what's going on in
the game so right now we're gonna I'm am
gonna hand right out at least the basic
you know drawing of a thing and then
when you start to build on this I'm
gonna do the copypasta because it won't
be anything new so so the first thing is
to draw we're just gonna draw basically
the game map we have access to game
information and so if you did like self
game info and then game info has map
size now how do I figure out all these
things right um there is a wiki for pi
sc2 but it's you know not really the
best vici you have ever seen and that's
okay no insult pie I really like pie se
to thank you for making it because I'm
sorry Python - SC - because Pius II -
was just impossible to understand so
anyways getting back to what I was
saying how do I figure this stuff out
honestly you can immediately like you
could go to the source code the other
thing you can do because a lot of times
in the source code things and Herot from
other things and blob on it gets really
hard to follow you could always just
print derp itself and just hope they
make useful names and you can see like
what are all the methods and just kind
of read through them or you can go to
the source code and compile the list by
reading it or whatever just know that
like that's just honestly what I do I
just go through there and and see what
my options are and that's how I found
game sighs I just figured it would be in
there so so what we're gonna say is game
data and this will be the the initial
image and we're just gonna make that NP
zeros and then an NP dot zeroes you pass
a shape so in the shape of our NP zeros
it's going to be so the shape will be a
tuple basically that will be self dot
game info dot map sup map size and then
one first if and then the zeroeth oh my
gosh what is Harrison up to what is he
up to
so in an array its height by width but
in the image its width by height so
we're going to reverse these because
we're using an array for an image and we
kind of want to flip them okay next
thing is it is not just X&Y they also
have color that took an embarrassing lot
embarrassingly long time for me to
figure out what the heck was going wrong
and then this will be our data type so
okay so that's game data now let's say
we want to draw stuff so let's say for
Nexus in self dot units of Nexus type we
want to draw them so we're gonna say the
next is let's say next position is Nexus
dot posts actually it'll be position
again how did I figure that one out I
printed out the Durr of it that's it so
and then I just read them and I was like
okay position sounds like what I want
and sure enough it was so that's next
pose so then we're just going to draw
this to our game data so CV 2 dots
circle and then in circle you say where
are we gonna draw this circle we're
drawing it in game data and then we're
on this game data are we gonna draw this
circle well we have to draw two intz in
CB 2 if I recall correctly so int and
that'll be next pose several and then
we'll do int next pose 1 and then is
that correct let me hang on to me
parentheses here okay so finally what is
the size we're gonna make it 10 pixels
it should be pretty large that's your
Nexus it's important and then what color
this is B G R so 0 255 0 that's 0 red
I'm sorry that's 0 blue full green 0 red
BG r Harrison ok and then I think the
next value is line width I'm just gonna
say negative 1 I believe that just makes
no line width okay so that draws
everything fancy that then what we want
to do is again so an array is height by
width we already kind of flipped that
around for ourselves now the last thing
that we need to do is so open CV is open
CV is going to work with 0 0 as like the
top left but coordinates are going to
work with 0 0 as the
bottom left thus we need to flip things
so we're gonna do flipped equals c v2
dot flip we're gonna flip that game data
on the zeroeth axis and then we're
basically done but i'm gonna go ahead
and resize it depending on your screen
resolution you might not need to do this
it's just I couldn't see it was like so
small on my screen I just wanted a
little bigger I can't see things so
resize CV to resize the actual imagery
that we're gonna save and use our neural
network on though is gonna be the
flipped image you wouldn't want to pass
resize there just be a waste of of data
really so anyway flipped D size none FX
2 and then F Y is 2 so it's like the
function of X and the function of Y it's
just we're just multiplying by 2 then
we're gonna go see B 2 dot M show and
we're gonna show it as Intel what are we
gonna show resize what are we going to
show it now just kidding wait ki 1
millisecond and that's it and we should
see some data let's run it and hopefully
we didn't hit an error we will see
and hopefully we get a like a pop up of
the CVT window and the placement of our
Nexus shoot let's see no petting it
where dude is it no is it not gonna show
up for me oh we never called Intel okay
yeah yeah awesome
so okay oh wait
self dot into okay Wow wait thank you to
my recent sponsor slash members they're
gonna change the name at some point Igor
Eduardo gg6 4d you owe to who you Najib
Siva fubá Jun Weng and Sam King thank
you guys very much you are all awesome
oh we air n peas not defined oh we
didn't import numpy as NP okay okay
we'll try again Wow Wow help up how did
that get there
oh my goodness whoa button beautiful
okay hopefully hopefully this works now
please don't crash that doesn't look
like a wanted okay okay this will be one
of our last errors i guarante i
guarantee it
so camelcase wait key because it's
python and that's how they like to do
things come on
this better work the people are gonna
start riding
got it okay so this is basically the map
right and then this green dot is our
Nexus placement on the map and then boom
we just set up another Nexus it looks a
little it looks a little down for some
reason I'm not really I think this is
just all empty space it might be like
trimmed or something in this map too it
looks like things are oriented correctly
we'll see if we get a third and Nexus
hmm and just in case I forget to mention
it one of the things that we definitely
want to do is if that if we send up
another one I'll show it
we're showing the Nexus on the map
before it's actually built before it's
actually ready you'd probably want to
only show it when it's ready
Wow this got rushed whoo he's gonna beat
us I'm pretty sure oh we did actually
get some void rays immediately see you
buddy yeah here you Joker
all right so authors the other Nexus so
it's building but we're already showing
its up that's probably a mistake yeah
you probably don't want to do that you
don't want to do it ready anyway we're
gonna do the rest of stuff in another
video because this one's probably
getting super super long if you have any
questions comments concerns whatever
feel free to leave them below otherwise
in the next tutorial we're going to
continue drawing beautiful pictures onto
our screen here for visualizations then
we'll save the data and hopefully train
a model that is better than random all
right
till next time