what is going on everybody and welcome
to part 7 of the halite 3 tutorial
series as well as the first part to the
ml section of halite 3 tutorials I do
want to stress I am here mostly just to
tinker with m/l I have no promises here
it's kind of like the Python plays GTA
or the sc2 series I don't know what the
outcomes gonna be it's just a realistic
example of me trying to solve a problem
with ml so in one case that was pretty
darn successful and the other one it
wasn't so so we'll see if you guys have
ideas how to make it better or whatever
feel free to leave it in the comment
section also if anybody has an ml type
of model that they just don't have the
processing power to go through if you
want you can submit those in the comment
sections too if I see something that
looks pretty darn cool we're interesting
all happily devote some processing power
to it to see if it's any good so if you
want to do that have at it
so with that let's just jump in and let
me talk about what I want to do and why
so the first thing I'd like to do is
let's head over to halite dot IO and
talk about the method I want to use so
one method that is probably gonna win
halite or at least be the best type is
you can parse replay files now I'm not
gonna totally hate on that method I plan
to use that method and I think that will
be the one that wins if there is a
winning ml method but I find that method
not like it's cheating but it's kind of
cheap it's not as cool to me as
something a little more raw so that's a
very supervised ml method and I think
it's way more interesting to do methods
that are a little less supervised so
what I would like to do is the type of
model that I always want to do which is
a more evolutionary based method
something that learns more from like
random is just fascinating to me I just
really like that so that's what I want
to do here and so anyways I know people
are gonna
like reinforcement learning this is
tunics that kind of reinforcement
learning but it's not cue learning which
is really what everyone means when they
say reinforcement learning anyway we
will be doing deep learning just I want
to do an evolutionary based model
because I think that's really cool if
you want to do a reinforcement learning
with cue learning example and again you
don't have a processing power let me
know post it below and I'll happily see
how good it is and I'll dedicate a
comparable amount of processing power to
it depending on how decent it is so and
give you credit and all that I won't
submit the bottom anything unless you
want me to anyways
let's watch a game did I can't click it
they're going so fast I can't click this
is super frustrating thank you
yeah I couldn't click on them okay so
here's an example of a game and here's
how I how I want to approach it in as
few words as less it's gonna be a lot of
talking I'm sorry there's a lot of
concept here so anyways so what's going
on here okay so in this case each player
has what four ships four or five ships
this I think green only has four yeah
anyway
each ship is its own little entity and
in the world of halite this game as I've
said before is really all about
navigation but the nice thing is in how
I to navigation was about you had a
vector and thrust and all that's like a
percentage is that was a very
challenging ml operation was in Halle 3
honestly this is a much more basic
problem to solve that doesn't mean that
competition is any less that we've got a
lot of the same players from last year
and the competition is just as high but
the actual challenge itself in the the
way you need to think that part's a lot
more basic you instead now you've got to
get a lot more fancy when it comes to
actual logic and stuff like that so I'm
definitely not dogging on this year's
game but in terms of navigation we only
have five options right you can only
move up down left right or not move so
that's pretty nice and it's a grid
structure so it's very basic
super simple for us to feed this
information to an AI or an MLA I and
specifically a convolutional neural
network it seems like a no friggin
brainer to feed it through a confident
so the only fear I have is they're
pretty small but you know we don't need
a huge continent we can use a smaller
continent and be just fine okay so I
want to feed it through a confident now
there's a couple of issues we have one
is the game sizes tend to vary a few
different ways we could overcome that we
could either use padding or we could
crop down to the smallest size which is
a 32 by 32 but if we're gonna do that
the other option is to instead of taking
the entire game map we can take things
relative to a ship now we're already
gonna need to do that so consider you
know I'm gonna zoom in but consider you
know this is your game map let's say it
was a smaller game Maps this one's go
already kind of big at 48 by 48 what
where should this ship go mmm this ship
could should clearly go left right or
west to get that halite spot what about
what about this ship should this ship
also head to that position probably not
this ship should probably maybe even
come down here cuz over here's a little
close to the enemy because it rolls over
well it's actually not close to the
enemy I'm zoomed in I forgot about that
so maybe he should go over here cuz he
is closer than the enemy is to this spot
right
but regardless each ship is relative so
each ship should have a relative
viewpoint as well so whether you wanted
to use the full map or a slice of the
map or whatever you want to not have
this let's say you're this ship you
actually want the map to look like this
and let's say we're at this ship we want
the map to look more like this right
with your ship the one you're
considering at the present moment in the
center unless you were going to for like
a macro model of some kind to figure out
okay where we send clusters of ships or
something like that which is totally
valid but what I want to do is an
evolutionary based model and I want to
do it on a per ship basis so each ship
the model will run a prediction for each
ship as we iterate through maybe later
on I'll change that but to start that's
kind of what I want to do
so then the next thing I want to do is
we need to get all of the information
around that ship so we need to get this
coordinate in this coordinate and this
one in this one and just do that around
that entire ship so now let's talk about
how I plan to do that because the good
thing is we have relative and game map
coordinates built into that little
halite API for us so doing this is
really just a logical question it's not
going to require too much digging on our
on our on our end so I'll just minimize
this will come into here and for now I'm
going to copy pasted sent a bot I'm
gonna call this testing grounds open
that up and let's just talk about how we
can get those coordinates so first of
all we just kind of want to get like all
radius I'm not gonna get a circle though
I'm gonna get a square so radius is
probably the wrong word to use so I'm
gonna say size let's say the size we
want is three so three in all directions
we'd like that square how would we do
that well what we all we really need to
do is like for I in or actually let's
say for X in range of negative one times
snot three sighs let's make a dynamic
here and then you need to go up to size
but the weight range works is it starts
from a point and goes up to not two so
we want to say size plus one so if I
print X here we'll see we go from
negative 3 to positive 3 so if we have a
three by three since because of zero
right the center position it's not going
to be a three by three and because it
goes from negative to positive it's also
known as six by six this will generate a
seven by seven okay so so then we could
just do the same thing for y so let me
just copy come down here paste and then
for Y and then we just print plus four
now we'll print XY and this should be
our relative coordinates at least it's
every combination of relative
coordinates but it is that what we want
well looking at this I
I would say no because this this was
this would like if we were to
pictorially represent this like so if we
were to get the halide information and
then pictorially represent it it
wouldn't look like this because of the
way we're iterating and I think just in
case someone's not understanding it what
we definitely what we want this is gonna
be the most beautiful stuff you have
ever seen I'm gonna draw a bunch of
these you'll have to excuse the mark I'm
not gonna waste too much time but I this
is something it's always like whenever I
have stuff like this I always have to
look at it visually so here let's say
our ship and let's just do let's say our
ship is we'll go with green and we're
right here this is our ship
mm so ship set Center then like we want
the square so since it's a 3 in theory I
guess we want like this this is ugly we
want this square though but what are the
coordinates at these other locations
well so this is up 1 and I want to say
like up is actually negative right and
then left would be negative as well so
at this point here this is the same X it
seems to be a little bigger that was my
phone not your phone this would be same
X so at 0 but then the Y would be a
negative 1 and then this would be again
same zero but negative 2 and then this
would be 0 negative 3 now if we continue
this would still be a negative 3 y but
this would be a negative 1 negative 3
and this would be negative 2 negative 3
I know it's running over but you get the
idea this will be a negative 3 negative
3 and so on so we know that this top row
should go in that sort of order and then
as you pop down here X here is still
negative 3 but now Y is negative 2 and
then this would be a negative 2 negative
2 and so on it should just continue with
that pattern
and right now that is not the pattern we
have we have negative three negative
three but then it's a negative three
negative two which is not correct right
it's going this way and we don't want
that we need it to go the other way
because we again it's imperative that we
keep how it looks pictorially because I
want to use a confident so continents
are gonna go based on patterns and so if
you have a cluster of halite chances are
we that's where the the ship should go
but if we don't go in the correct order
a continent might wreck might still
figure it out but it's less likely to we
need things to be in the right order and
in fact it might even actually flip
things but this is exactly why at least
for me personally whenever I'm dealing
with arrays I it needs to be visualized
I need to see it visually to confirm
that my theory is correct and this is
like one of the very few times I ever
actually write stuff down to test but
anyways this is not what I expect so
what's our problem well we're in the
rating by X's and not by Y's right
because if we want to iterate by Y well
we need to Y needs to come first so if
we should get away with just flipping Y
X run that again just go up to the very
top okay negative 3 negative 3 negative
2 negative 3 negative 1 negative 3 right
we're gonna stay at negative 3 until
we're done with all the negative threes
then we're gonna do all the negatives
and so on so it's going to go from the
top down perfect that would generate
almost an image we need to like since Y
is our Rho Rho is just an empty list
here let's just say surroundings and
then we will not do that we'll do this
and then yeah we would say Rho dot app
and XY so we pinned the rows and then at
the end we surroundings surroundings dot
up and the row and at the very end let's
print a rather than even print
surroundings as for our in surroundings
let's print our and now let me delete
this print run that negative three
negative three and then all the negative
threes then all the negative 2s and all
that now if we were to grab let's say
halide amount at each of these locations
it would start to look something like
this does right if we displayed
highlight amount by a size of in color
of the square okay we could definitely
do that and it would look exactly like
that okay so we have the rep hey man
becuse we have the relative coordinates
now what we need to do is get this for
every single ship okay and probably have
a larger size so how do we get this for
every single ship well I believe it was
in the regular tutorials I'm pretty sure
we covered this you can take a current
position and then add another position
to it even if that's a negative so it
would subtract or whatever so for
example let's just go - lets go Python
programming net come down here and we'll
go to how light 3 wow I think I just
spotted a bug with that add I'm gonna
have to check that out we check that out
oh my gotta go back yeah
let's see yeah that's clearly an ad in
the middle it won't have to make sure I
disable ads on this page bought
tutorials ultra I have this this is like
a newer tutorial page and I think that's
why that's happening anyway back to work
let's go to the final tutorial just grab
the code we actually probably don't have
to go this deep into the tutorial
because we actually just we just want a
skeleton here but I'm gonna copy the
whole thing anyways and then we're in
sent to bot paste the whole thing escape
out of there and then what I want to do
is let's go to the very tippity-top
we don't need ship states anymore let's
change the name as well sent a bot - you
know let's key we keep the command Q
keep Direction order that's fine we
might want the
later position choices we don't want
that anymore we're not gonna use that
logic well we expect our AI to figure
that out now because basically what what
I would like is the AI to control
everything everything possible though
the one thing that we could have AI do
but it we'd have to be a separate model
is when should you create more ships and
when should you stop creating ships we
can we can definitely do that but it
would be a different model because this
is a micro model and create ship as a
macro choice so but in terms of when the
ships are collecting or depositing and
where they're going anything every the
AI is choosing where that ship travels
period so for shipping media I guess you
okay so actually want to delete all this
logic really everything will keep one of
the command queues okay cool so okay so
the first thing I'm gonna do is let's
just delete all the way in these extra
so now it just it just moves the it just
holds still the ship basically and then
the other thing is if Len meet I got
ships let's just only make one ship so
if it's less than one and we have the
highlight to do it let's make a ship
also this should be a constant its
constant dot i think it's ship
underscore cost will find out okay so
all this does is just iterate over the
ships okay and doesn't move them and we
will only make one ship so the one thing
i want to do well we got a few things
first of all we can take this code so i
don't need to do this a little bit here
but i do want this bit here so i'm going
to copy this come into here and i'm just
going to paste it in here because we
we're gonna need it at some point we
just don't need to write this instant
amen now I'm gonna tab that over okay
and okay now what I want to do
is I want to confirm a few things
because like I was saying so like here
these are relative coordinates but what
we would like to eventually be able to
do is query the halite map itself for
these values we want to be able to ask
the highlight map okay how much halite
is there also is there a ship on that
position is it our ship is it an enemy
ship is there a drop-off or a Depot
rather or no it is drop-off I keep
wanting to call them depots a shipyard
or whatever is it our shipyard is it
enemy ship all that stuff we want to be
able to query for all those things so we
can't pass to the game map negative 3
negative 3 that's just I mean we could
but it would be the wrong position we
actually need to pass the ship's current
position plus those relative coordinates
so the way we do that coming back to our
code now would be something like this so
we could say let's just do a logging
info and then we'll use an F string and
then let's first let's print ship dot
position that's the current ship
position then let's print let's do a
ship dot position position plus and then
a capital P position so we have to
convert it to a position type let's do
negative 3 3 so that's the current
position the current ship's position is
the true position on the game map then
we want to pass that relative position
that should just that should correctly
modify the position by that'm relative
amount basically so then what we're
hoping is that we could also query the
game map by that exact thing so let's
just make another logging info another F
string and then we'd like to be able to
just directly query game map and then in
here you put the coordinates the actual
physical coordinates on the game map so
my expectation is that we could actually
take this right here pass that as the
coordinates that's our hope and if this
works we'll be able to see a couple of
things one it will tell us like if we as
we pass through like 0 my hope is that
if I pass a negative position
there is no negative position on the map
it just automatically rolls us my my
belief is that it does but we need we
need to confirm that before we go down
the rabbit hole because pretty soon
we're never going to be looking at this
ever again so we definitely want to be a
hundred percent certain it's doing what
we want and also we will visualize it
even further this is a kind of bad
version of visualizing but I will
visualize it as well in something
probably open CV or matplotlib to make
sure what's happening is what we expect
because if because if it's not we're
wasting a lot of our time so okay with
that let's go and run it let me go into
here let's edit it is sent to bot by so
we don't have to change that then we'll
run that it looks like it's at least
running so it should be creating log
files for us looks like it is okay
they're both sent to bot so it doesn't
really matter okay so this was a current
position negative 3 3 so 25 23 and then
we come down here and we can see we're
definitely getting that map cell object
and it even tells us it tells us that
position also tells us how much halide
is there we can also query for is there
a ship there and all kinds of stuff like
that so so we we can figure out all of
that information that's working the one
thing we didn't check is if the ship is
moving so let's go ahead and move the
ship because what I need to know if it
crosses over zero if we're gonna be able
to handle that so let's instead say ship
north so we're just gonna every move the
ship just moves north nothing fancy
save that let's rerun that and then
we'll come over here
click on this okay so it's definitely
moving let's just scroll down down down
down until we get to a zero issue so at
this point it actually got to a zero and
it becomes a 25:42 and then it becomes a
25 - that's curious
cuz wasn't it no it's a 40 by 40 okay
that makes sense so 2 over is okay and
then so from 41 like 41 is out of bounds
so is 42 42 is out of bounds
so that rolls over and becomes a 25 -
okay so that does that does work it is
the way we intended okay great
so now what we have to do is begin
polling so rather than row append XY we
actually want to row append game map
position X Y and we want to actually
append three different things so we will
have a three-channel
continent it won't be RGB or BGR or
color at all it'll be you know halide
amount ship amount and or ship let's say
ship and if it is if there is a ship you
know the intensity could be how much
halide is on that ship or ship yard
right or drop-off okay and we could even
have is a ship at the drop at the
shipyard which would be curious actually
we would already know that from the
previous one so yeah so it would just be
halide amount ship shipyard and if it's
an enemy ship we could just say negative
one times that enemy ships halite amount
if the enemy shipyard negative one times
that shipyard done so I think I'll stop
it here and continue in the next one
we'll do that we'll visualize it and
maybe even start building maybe some
random data I'm not really sure how far
we'll get I'm gonna try to keep these
not too too long but also not too short
but yeah if you guys have suggestions
whatever
feel free to lean below again if you if
you make one or you've got a different
idea and you want someone to test it out
with some some honkin processing power
let me know in the comments if you want
me to run it and see how far we can get
totally willing to do that also quick
shout out to my most recent channel
members you day Dave John Walton and
Bing Lee thank you guys very much for
your support it allows me to do stuff
just like this which is a super fun and
it's an awesome job so I really
appreciate you guys's support so anyways
that's it for now
like I said next video will visualize
make sure what we're gonna collect is
exactly right we'll make sure it looks
like something a confident could
actually learn from then we'll start
building data and all we're gonna do is
build data off of random moves and then
we can slice away the best you know
games like we could go based on who won
or something else slice away those and
then train a model and then iteratively
just keep doing that and see just see
how far we can get I have a suspicion it
won't be super far but I'm really
curious to see how far so anyways that's
what we're gonna be doing in the next
tutorial I will see you guys there