[WOODEN TRAIN WHISTLE]
Welcome back to Part 2 of my video series about Teachable
Machine and P5 ML5.
So if you're wondering, wow, where am I?
You probably should go back and watch the first video, where
I look at what Teachable Machine is, how to train an image
classifier in the browser with Teachable Machine,
save your model, upload it, and then bring that model
into a P5 JS 5 sketch, and the project that I did
displayed an emoji that corresponds to the label
that the classifier classified in the image,
like a unicorn, a rainbow--
you know, that kind of stuff.
But what I want to show you how to do in this video
is how you could use this technique
to control an interaction.
And a good example of that, I think, is a game.
So I have coding challenge number something,
which was the snake game.
So here's the version of my snake game in P5,
where I have to drive around, and I can't--
I'm so bad at it--
pick up the food, and I can play it.
And there's really only four things I can do--
up, down, left, right.
Well, guess what?
That matches up with a classification problem.
What if I showed an image for left--
left, an image for right, an image for up,
and an image for a down.
So I'm going to go and try to train a model with images
for all of those.
All right, here comes left.
Here is right.
I'm going to add a class for up, but I
need to use both my hands for up,
so I'm going to turn off the hold to record,
and I'm gonna give myself a countdown.
Then I'm going to add a class for down.
And now I really need a class for do nothing, right?
Because it always have to classify as something,
so what if I just stand very still right in the middle,
that'll be the do nothing.
And we'll call that nothing.
All right, so I have the data for left, right, up, down,
and nothing.
Let's train the model.
Let's test it out.
OK, nothing, right, left, up, down, up, right, left, nothing.
All right!
This is awesome!
Kimberly, it worked!
Let's export the model.
Now that I've uploaded the model and I have the URL,
I want to use it in my code.
I have my snake game, which I wrote in challenge number 115.
I also have the code that I wrote in the previous video.
So rather than write all that again, let me just copy paste
over the bits that I need, and I'll talk you
through what those are.
So certainly I need the Video, Label, and Classifier
and Preload functions.
So let me bring that into my snake game.
Then, from set up, I need to create the video
and start classifying the video.
So let me bring that in here, and I'm
going to do that right after createCanvas.
I need this Classified Video function.
And I need this Got Results function.
So even just running this to see if I don't get any errors-- oh!
But I got the error that I forgot about.
So because I started with an example that
doesn't have the ML5 library imported,
I've got to manually import it into this example.
And the way that I do that is by copying and pasting the script
tag to index the HTML.
Here's the script tag that I need on the ML5 JS home page.
And in my snake example, I can reveal the index of HML
file over here on the left, click on it,
and add in that script tag right there.
Save and run again.
No errors, I don't see anything, but it's
classifying the images.
Let me draw the video so we can see it.
So there's the video.
Let me also draw the label, so I can see what it is.
So I'm gonna say text label 10, 50.
And let's just make it text size 32, Fill to 55.
Let's do this.
And it's a train.
OK, I've got to get my model URL.
That's the model URL from my previous example.
Copy-- and again, your URL might be slightly different.
Look at what it tells you.
OK, I've got the new one.
Run it again.
Something interesting is going to happen here, I think.
Nothing, that's good.
Up is good.
Down is good.
Up is good.
Down is good.
Right and left seem to be wrong.
Well guess what?
My image is not flipped.
So Teachable Machine, if you go back
to look at when I was training, when you saw my hand over here,
the image was mirrored because I'm looking at the computer
like it's a mirror.
So what I actually want to do for this to work more
accurately is flip the video.
And fortunately, ML5 has a nice utility
for doing this-- there's a variety of different ways
I can do this with P5 only but M5
knows that this might come up, and can flip it for you.
To take care of this, I'm going to add a new variable.
I'm going to call it flipVideo.
I'm going to say, right in Setup,
I'm going to say flipVideo equals ML5 flipImage.
This is the function that takes any image and flips it
and puts it in the new variable.
I'm going to need to do this actually continuously,
because it's a video-- there's always
new images, and every new image, I need to flip it.
So I'm going to put this right here in the classify function
as well.
To make this easier to see, let me also make the canvas
the actual size of the video.
Then I'm going to classify the flipped video,
and also draw the flipped video.
And now let's see what happens.
I can see the full video.
And, also, it's mirrored.
So that's left, that's right, that's up, and that's down.
Left needs some work, but maybe I
need to do better training data.
Now that I have those labels, guess what?
I can control the snake based on the labels.
The way that I did it before was in the keyPressed function,
with key codes.
Let's change this function name to controlSnake.
We'll change keyCode to label, and instead of the arrows,
we'll say the label names.
This was something extra that was in the example
before, that I could just make a snake get bigger for debugging,
so I don't need that anymore.
I just want to control left, right, down, up, and then
if it's nothing, don't do anything.
So I don't need to add that to the "if" statement.
In gotResults, once I get the label, I can call controlSnake,
and we're off and running.
Up, go up!
There we go.
Go the right!
Go to the left!
You got it!
Get that food!
You got it!
No, go to the left, left!
Up!
Right!
Down!
Left!
Down, right!
Yes.
Ohh!
It works!
I don't want to do it anymore, because that
was as good as I could get it.
But guess what?
I bet you you could make a better version of this.
Train your own model with your own images,
bring it into the snake game, bring this to another game,
share it with me in the comments, remember--
if you're doing something where left and right really matters,
you're going to want to flip the image,
just like it's flipped in Teachable Machine.
Thanks for watching.
I've got one more video for you, where
I'm going to show you how to use the sound classifier,
and that's coming up next.
And who knows?
Maybe by the time you're watching this,
there'll be even more videos.
Thanks for watching, make your own creative version,
and I'll see you in a future episode.
[WOODEN TRAIN WHISTLE]
[MUSIC PLAYING]