[Music]
hello good morning and welcome to the
coding train with me your host Daniel
chip I always feel so ridiculous doing
this I you you may have noticed that I
wasn't around last week live streaming
that was because I was sick I had
another lovely case of laryngitis but
things think to have come back I'm able
to speak I'm gonna try to limit myself
to just a couple hours today which is
really how much I should limit myself to
always but sometimes things tend to
spiral out of control and I'll suddenly
have livestream for four hours wondering
why but I will be back tomorrow
so today whatever I do today will be in
two parts so part one today and part
what to tomorrow
so first let me thank today's sponsor -
Lane - Lane is a password manager I
could not live without password
management and it's something that you
if you don't use a password manager you
might consider checking out - Lane and
you can go to - line.com slash coding
train to get a free 30 days of - lien
premium and also a discount if you
choose to purchase it so I'll come back
and talk a little bit more about - Lane
later but I want to thank them right
from the outset for helping make it
possible what I do here on this YouTube
channel I also want to talk about
something near and dear and close to my
heart so let me whoops that doesn't
matter
let me open up a web browser by the way
this is a new computer and so I have not
done anything really to set up this new
computer I mean I set up my own personal
log into this computer with my dev
environment but then I this morning just
created a coding train login and I've
not set up the dev environment so that's
something I will actually do during
today's livestream which will either
thrill you for it because you'll get to
see some of these you know behind the
scenes it shouldn't be behind the scenes
it should be right out in the front of
the scenes but it's a little bit of you
know you might be wondering like well
what are the visual studio code
extensions you have installed and that
kind of stuff so I'm going to go through
that because I have to can't do anything
without having gone through that and
then that's one thing oh but where I was
I was talking about I want to talk about
the processing foundation so few things
about the processing foundation so if
you're not familiar with the processing
foundation processing foundation is a US
not-for-profit 501c3 tax-exempt
organization we are a charity based in
New York State if you're wondering about
all the legal jargon processing is the
entity that maintains these software
applicants processing p5.js processing
dot PI or Python version and processing
processing for Android and then there's
also it's very confusing processing for
pi which is the Raspberry Pi
implementation of processing so all of
these different tools maybe you use them
maybe you teach with them maybe you just
been exposed to them because you watch
them on the coding train whitebalance
looks good on green screen maybe the
folk
sorry I business it appears that if I
cannot I cannot keep my focus which
might actually be true but what happens
is I'm on a little thought and then I
see out of the corner of my eye I had a
message it seems important I start to
read it and then I lose track of what
I'm talking about and here we are let me
double check the focus
today I will be advertising the mud
coffee let's put that here and see if I
can focus on this coffee because it
seems like I might be out of focus a
little bit then I will come back and
talk about the processing foundation oh
yeah it does seem a little out of focus
let's see let's see what that better
maybe that is better tell me everyone if
the focus seems a little better on me
now hopefully it does and he'll have a
sip of my bud coffee
it is not a password manager but it will
keep you awake I've been having a lot of
trouble sleeping I think I have to I
think I really need a no screens policy
after 8 p.m.
I've got a little bit of a Pokemon go
problem
should I not show you admit that or no
processing foundation so I have been
part of the processing project as a
contributor or enthusiastic user of the
processing since around 2003 when I
learned about the project from some
workshops that others had taught at ITP
where I am currently and over the years
I've been you know with each year that
passes I've got a little bit more
involved maybe to get less involved I
don't know where the what the exact
trajectory is but without the processing
foundation and all of and countless
hours of volunteer work and some paid
work that's been generously funded
through donors and other types of
initiatives none of what I do on this
channel would be possible so recently
there is a new one of the one of the
tools that I use probably the most these
days is p5.js library so I'm going to
the processing foundation medium page
which I probably could have got to from
from here there we go I might have gone
to the wrong place there oh this is what
I wanted to look at but I wanted to look
at the East too so first of all if
you're wondering what's the latest stuff
going on with the processing foundation
you should check out the the articles
that are on medium and the first one I
want to highlight for you is this
article written by Lauren McCarthy the
creator and leader lead developer of the
p5.js project and I won't read
the whole article for you but you can
kind of get the idea here no no I don't
need to sign in you can kind of get the
idea here from the title making space
for the future of p5.js
so after seven years of developing and
leading the project lauren is planning
to transition out of her role as project
lead in January 2020 and we are hoping
to have a rotating model of leadership
with a new leader every year and this
would be a paid position so this is
something we are currently act at this
is something that we are currently
actively fundraising for because this is
something that is a question of can this
project even continue how do you sustain
a open source a healthy inclusive open
source project and this is a constant
struggle and there are no easy answers
and I'm sure lots of projects run up
into all the same questions and issues
lots of people are working and thinking
about this I encourage you to read this
entire article written by Lauren
McCarthy and if you want to if you have
the means and Ken you know so part the
reason why it's an open free open-source
library is some of you watching this
maybe cannot support the project
financially and that's totally fine
that's great that's what it's here for
but if you can or you know somebody who
can or there's a way that you can
contribute or advocate for it in some
capacity I would encourage you to donate
and so I want to draw your attention to
this other article by Dorothy Santos who
is the project manager of the process of
sorry program manager of the processing
foundation and and we are running a
fundraiser like a month I don't all the
information is in this article written
by Dorothy you will notice something
here that might look familiar to you I
don't know maybe you've seen more rows
and the sort of like a little booklet or
zine this looks like something that you
might have seen on the coding train it's
because as part of the fundraiser there
are
okay no I don't need to be interrupted
there are rewards rewards from this is
work by the artist Maya man these are
raizo prints that you can get as part of
your donation Kate Hollenbach and I'm
gonna scroll down and find Saskia uh
freckie she's told me how to pronounce
her name I've drawn a little bit of a
blank right now but I think it's for
these I love I love all of you but I I
have a affinity to these kinds of
geometric patterns that are created with
processing and p5.js and other but sigh
who is the coding trained community
manager who many of you may have come
across in your communications with
things around this channel of has been
working very hard on a coding train zine
so as of now this is exclusive to
donations to the processing foundation
you know I will share all the code and
all the images in here are part of
things that have been on the channel so
but if you are interested this is
something that I am working on producing
and could be part of a reward with a
donation I will also be doing my annual
processing fun livestream fundraiser
holiday fundraiser closes not tuned what
playing a ukulele and making a fool of
myself singing holiday songs with coding
themed lyrics I will be doing that next
week probably a week from today
that's the date I'm targeting I haven't
in scheduled yet oh but wait next
Thursday or Friday or Wednesday so yeah
I can't do it after the 20th I'll be
away so that's my plan so actually
speaking of which I need a maybe I don't
know if violet is watching violet has
been working on helping to maintain a
lot of the community contributions and
github repos associated with a coding
train they are doing they're being so
it's I'm so grateful for their help with
this and if I go to github.com slash
coding train I'm trying to think like
where where where can you submit didn't
I do like let's see holiday
I feel like there was a repo last year
for this having trouble remembering
let's look oh wait I can type in here
holiday holiday songs okay we could just
use this still so I had a book of like
holiday ukulele songs which were the
songs that I had chords for that I could
play I saw a YouTube tutorial about how
to play last physical last Christmas I
don't know about this Christmas thing I
never I was not somebody who celebrated
Christmas as a child you can make your
inferences as to what sort of winter
themed holidays that I my family
celebrated Hanukkah I don't know why I'm
dancing around the Hanukkah bush but but
I love I have a love for holiday music
as embarrassing as that might be and so
if you have some songs you want to write
the lyrics for provide me ukulele chords
for or a backing track I will play about
b3 backing track even you can submit
that to this repo please I encourage you
contribute to my time of making a fool
of myself next week okay now so I've
talked about the processing foundation I
have talked I've thanked dashlane ah one
other thing I want to mention let's go
to their might be other things I want to
mention but in my update on what's
happening in the world of the coding
trade Oh
two things I want to mention so i one
thing that i did recently
hmm am I gonna be able to find this wait
no okay
I'm looking for here we go so I recently
had was lucky enough to have my
beginners guide to machine learning
playlist converted to a youtube learning
playlist this is a new feature I don't
know if it's out of beta or still in
beta but this is a new feature within
the last Kevin Ware's launched but
sometime in the last year
which is allows for a little bit of
extra organization of video content on
YouTube in with more like a syllabus
style if you will
organization so you I just want to like
a lot of there's nothing there's not
well there there's some pretty new
content here because I some of these
videos just came out within the last
week but what this is whoops is I've
basically collected all of the videos
where I'm using the ml 5gs library
including my sort of introduction to the
library the image classification videos
the transfer learning videos this is a
little bit of an aside but it's useful
the canon classification videos the
teachable machine videos the train your
own neural network videos which is what
i'm actually continuing today they'll go
in a new section probably called pose
poses working with poses i'm going to
call it and then i a few different
videos which touched on audio so they're
here so you know if you are looking to
sort of find and go through somewhat
sequentially all of the javascript
machine learning videos i have they're
here now
that's to say it's not actually all the
JavaScript videos i have because there
are a ton of them if you UPS know ah oh
go come back pull my log in as i don't
even know what's going on sleeping - who
can you hear that
you cannot hold on everybody we're gonna
have a little like random number ASMR no
no we're not we're not I got to keep
moving here
welcome back to that later great channel
here by the way every day I should check
out every day I what am I looking for my
home my channel what's going on your
channel when my logged in ass no I'm uh
oh I'm somebody else with no channel I
have too many Google login it's such a
problem this is my Daniel at the coding
train calm login which you'd think would
be the owner of my channel it's not some
day I'll figure that out dark theme is
on
YouTube this is a log and I only use for
logging and during my livestream so who
knows what I'm subscribed to or what is
recommending but how why do I not see
the sidebar oh wait I do this there we
go Oh
Oh No I've saved this I guess oh no this
is what I'm looking for oh this is the
worst what I was saying is I have a lot
of other videos on neural networks and
machine learning but thank you David I
will mention that there's a lot of other
videos on machine learning they're a bit
lower level and deal with a little bit
in Python but mostly like programming a
neural network from scratch and some
other background math and that sort of
thing so okay by the way this spring I
I'm only just take your temperature on
this I'm not gonna do a poll not gonna
do anything scientific here but I am
thinking of redoing this entire series
which was recorded let's see let's look
at one of these like vector video so
first of all look how different I look I
mean do you see any gray hair where's
those stripes in the beard they're gone
so this was recorded quite some time ago
it was published on YouTube in 2015 but
I think I even recorded these before
that so it's 2025 years later I am
thinking of redoing all of these videos
in JavaScript along with writing
updating the nature of code book so if
that's the thing that would excite you
let me know every day I can't we c'mon
that is correct
and again if I apologize for this for
those of you in the member slack channel
if you could remove the previews I can
see very little of the history of the
conversation I can see none of it as
soon as there's like a big unfolding
preview of something ok so there were
other things mentioned there's the wheel
there's the community contributions I
think those were other things so let me
mention the wheel we're actually before
I mentioned the wheel David maybe you
could point
where I should go where's the best place
for the discussion on that and let me go
to community contributions so I know
Simon always points out the videos that
I never showed any of the communities on
at this point I'm probably not gonna get
to ones that are a big backlog and lists
or something that I particularly want to
highlight but I will at least show you
the community contributions on the most
recent coding challenge which was
released yesterday or two days ago where
I created I implemented the minimax
algorithm to have a to play the game
tic-tac-toe so let's go to this first of
all let's go to the live demo where I
will now attempt
to beat this player this computer that
sick tactile by way you could like do
some kind of cool gesture tic-tac-toe
game haha see that move watch this now
I'm gonna go here haha here cannot get
me okay got you oh oh so sad sorry
coding train github page it's the most
recent repo created so that was the
challenge the X's are being played by
the minimax algorithm if you're
wondering what that is there's a
30-minute video for you to watch and
then there have already been Wow
I have not been I did not look at this
this morning there are one two three
four five well two of them are from
Simon so five unique people
contributions already there's
tic-tac-toe minimax in Python with alpha
beta pruning alpha beta pruning is not
something I implemented as part of the
challenge I alluded to it you know I I
assume that this is not something that I
can easily run here but I would
encourage you to check out there's a
David Snyder's example of this in Python
it's nice this is wonderful anytime
you're creating something if it's not
something that just runs in the browser
natively making a little gift including
this part of readme is a wonderful thing
to do
thank you for that I'm going backwards
for whatever reason reverse I ooh Oh Oh
seriously Wow this is reverse I'm in
Emacs with alpha beta beta pruning by
copper France all right everybody ready
for me to show my reverse eye skills Oh
we stretch is it reverse i reverse see
live LC new live LT I don't know am I am
I going first what are these little dots
that I don't I haven't played this game
in a very long time oh okay well I guess
I'm black
is it giving me a clues of where to go
or those are the only places I can go oh
there's the only place I can go by the
way I'm not thinking at all what I'm
doing so please okay I don't think I'm
gonna win this feel like oh the guy
remember I played this in years the goal
service to get the corners right
reversed see I'm being told oh Kobe says
their contribution is still waiting to
be approved so Kobe I can maybe pull it
up anyway all right all right
whose is incredible I'm gonna play this
on my own tie fighter thank you copper
France for this I'm very curious to take
a look at your code do something about
my look at this wow wow so this is
actually it looks quite similar looks
like the whole game has been implemented
here amazing okay let's keep going
and go - okay so Simon I know cuz Simon
made his own video about this where he
made a chess AI I'm just gonna go so so
one of the things about the minimax
algorithm is with tic-tac-toe I always
looked into the tree of possible
outcomes all the way to the end of the
game but there are some games I
mentioned this in the video that are so
complex you couldn't eat computationally
visit every single possible move that
could come from the from now to the end
of the game so with something like chess
you have to decide well whatever your
heuristic is whatever your estimated way
of computing the sort of score of the
game at any given time
how far down how many moves ahead do you
want to look so I this second one from
Simon has a customized a little search
depth but let's just look at the first
one and one thing I'm curious about
Simon is did you implement your own
version of chess or did you use like an
existing chest library uh-oh Loui mouth
shall I am very sorry
this one a my French is terrible and I
shouldn't have done that
tried to pronounce perversity in French
tried not even a French or the origin
game okay I'm curious did you implement
your own version of chess or did you
find
an implementation of chess in JavaScript
and then just add the minimax algorithm
to it so we can see here that I assume
if I do something like this and move my
pawn and then move my other thingy you
could see my chess expertise I'm gonna
lose definitely gonna lose this is wild
amazing to see Thank You Simon encourage
you to play this I'm sure some of you
who are sophisticated chess players will
easily beat this AI but it is
interesting to think about house house
how good at playing the game can minimax
be with chess let's go to tic-tac-toe
with alpha beta pruning so this looks
very much like whoops this is very
similar to my game and if we look at the
code whoops
source code what I'm curious to see is
it really easy to see where the alpha
yeah so you can see here in with the
they function has for in addition to the
depth has additional arguments alpha and
beta which are keeping track of what the
score is and allowing the algorithm to
skip down certain paths down the tree
knowing that there couldn't possibly be
a better outcome there so Simon is
saying that he used chess j/s for the
chess engine and chessboard JS for the
visualizer that's great so I wonder if
that's something that I could actually
tackle it's not realistic I think for me
for me to program my own chess engine
not without like a very long multi-part
series but certainly certainly doing
something like what Simon did could be
possible all right let's see if we can
find Kobe's contribution oh it's already
added Connect four Wow
so Kobe looks like he added
in emacs connect 4 and minimax connect 4
with alpha beta pruning once again I'm
just gonna go to the regular connect 4
this is something I was actually hoping
to do I'm gonna move here whoops whoops
back it's a little bit slow there that's
not where I meant to go let's go here do
not want to give them Oh weird
not what I expected ah no no bad
I shouldn't go there that's bad move all
right all right all right you get the
idea everybody you've all had better
ideas of what moves you dude this is
great Thank You Koby for this this is
pretty amazing it even just that basic
algorithm I will the normal version is
really slow showing the alpha beta
pruning zoos so this would be so I guess
from I'm gathering from Kobe that that
Kinect for is a good example of there
are so many more possibilities than
tic-tac-toe that having the alpha beta
pruning really improves the speed so I
that would be something that's worth me
doing a follow up perhaps about yeah I
can it feels much more responsive and
quick now of course I think I just lost
so thank you alright
and then so thank you for those
community contributions please continue
to make them if you don't know how to
add one there's some instructions and
and and information here just file a
github issue asking for help I will say
that don't be intimidated by this
amazingness and sophistication of these
I want your community contributions even
if all you did just change the colors of
what I made to make it your own
beautiful like rainbow themed
tic-tac-toe like I want to see
everything that you do don't worry about
getting things wrong or it's not cool
enough not it's all wonderful in my eyes
that you're watching these videos and
making stuff and in particular something
that some of you who are maybe I have a
affinity for visual design or interface
design you know you might I would
welcome versions that clean up how the
interaction is and change the way the
game looks visually and those are
exciting and wonderful to see okay so
last update before I start actually
doing some stuff here always good by the
way I uh I think it's so funny that I
always just type github flash website
I think that's gonna get me somewhere
coding train I don't know I don't know
where I'm going just typing get up calm
slash website coding train if I go to
repositories it is the most recent
topics wheel so let's see
so David Snyder's not a lot here right
now but big boots asks sorry to come
back to just answer this question big
boots asks hi big boots does it have to
be p5 to contribute absolutely not I
think though the whole reason for having
the contributions is for people to bring
to it something that they are excited to
share and that might be I made a version
of this in a different programming
language that's wonderful
the advantage of p5 is yours can be run
easily in the browser but if yours
doesn't run in the browser link to a
video or a gif or some other kind of
documentation if you can so it's easy to
take a look
darshan is asking me something about the
community contribution for the
regression challenge but it didn't work
for an audio context problem hmm
ah oh you mean it didn't work last time
but now it does work
sure darshan thank you for your
contribution I will take a look at that
where where do I the regression channel
the regression video and
regression based on node frequency and
color predictor okay I don't know what
to do so something to think about
is to think about what that may be I'm
just gonna figure it out oh this is very
similar to my si si si ah this is just
like my example so of course I know what
to do oh and I get to like just draw
stuff around that's so much better
let's train it so I know oh I guess the
stuff what happened this time I tried
do we still have the audio context
problem I'm sorry please everybody check
this out I'm sure I'm just completely
incompetent and not getting difficult to
work correctly but when they almost say
so one thing that I think is something
for all of you to think about when you
create interactive work to put on the
web is you have to think about what
context people are coming to it from now
I know what keys to press because this
is based on the challenge and in fact
that's this is great this is being
shared as a variation off of that
example and if presumably someone's
checking out has watched the video and
so but the thing you could think about
in terms of always having on the page
some instructions or user test it with
somebody a friend before you post it not
have a thing could always help at least
for classes
ah okay but try this one more time c d e
f i've got for train alright we've got
to los people we've got a loss okay all
right I think it's trained is what am I
supposed to hear something oh it's the
color oh that is so cool oh I see it's
not sound it's color that is beautiful
looking I can paint now I love this so I
feel like the letters are kind of
getting in the way of my experience with
this because I love the fact that it's
now this basically gradient over and did
you do a regression with three different
outputs you must have this is beautiful
I think there's so much potential to
this I'm sorry that it took me so long
to figure it out and so I would think
about maybe what you might consider
doing is instead of having the user type
in these letters which is I res what I
did in my example but maybe there's a
way there could be a color picker here
and I could paint certain colors and
then I could train the model and then
paint over the space based on those oh
there's so many possibilities and image
processing and filtering and interact
our this is wonderful thank you for
sharing this thank you for that I get it
right
I really enjoyed I really I really think
this is wonderful
great work if you make if you update it
or make any changes to it please let me
know thank you darshan Belfer darshan
okay and let me know if I mispronounced
your name by the way can I just say
something if I ever ever ever ever ever
get anyone's pronouns wrong please reach
out to me and let me know they make an
effort to be thoughtful about how I
refer to people on the channel but I'm
sure I make mistakes and I want you to
reach out to me and correct me and help
me if at any point I've mispronounced
your name get an incorrect pronoun refer
to you by the wrong name please reach
out to me any way that you know how to
reach out to people on the Internet okay
[Music]
are we at 10:40 we are so I've got about
an hour and 20 minutes left I think what
I'm gonna do is work on setting up the
dev environment and then take a short
break and start talking about and
teaching you about Poe's net in the ml5
library and looking out about how to use
Poe's net with the neural network
classifier so we do the ASMR random
number reading now though the one thing
I want to do actually cuz it could be
necessary later so I'm just gonna go to
I should do this oh no no no this is
this weird thing with a Mac where you go
to like audio MIDI setup and then I
create a multi output device because I
want to go to the speakers and pro
capture Pro capture is what is the
capture card that's taking the signal
from my laptop
so now I think if I were to go to
see what sort of meditation YouTube
wants to recommend to me today how come
you don't hear that
[Music]
no output multi-output device
now you hear that okay
[Music]
sixty nine thousand two hundred and
fifty one
fifty-four thousand four hundred and
sixty
60 1634
31
[Music]
seventy 1848
seventy two thousand two hundred and
eleven fifty three thousand okay perfect
that was me testing the computer audio
my voice is still a little shot doing
the best again I've got to really gotta
get a lot of vocal rest if I'm gonna do
this holiday song fundraiser next week
okay so let's see what I honestly like I
don't even know what's going on in this
computer let me open up I term so ah
okay I term is the terminal application
that I like to use one setting that I
really really really feel that I need I
think it's under it's the one that like
when you resize the font it doesn't
resize the window does anybody know
where that one is I also like to use
reuse previous sessions directory I
should really make a video about all
these settings not that mine are any
good parents I mean where is this one
advanced no profiles general text where
is this one I can never find this one
and he's under general
zooom but just window and changing font
size there it is hate that setting okay
because what I like to do is I like to
be able to increase the font size well
we've got a spam problem here in the
chat unfortunately oh thank you thank
you to the volunteer moderators or
inevitably gonna help out with this
sorry everybody
why why why people why can't we just
enjoy Thursday morning of coding live on
the Internet I'm gonna try to do
something about this as well I know like
what did something I do something to
prompt this there's so many people all
of a sudden is it like a some kind of
emoji army coming to DDoS what is going
on why does this happen I guess I could
turn me I should turn on slow mode IP
ban all right all right everybody you'll
get to enjoy the this dot song when I
figure this out as always I always
forget to dis stop this stop it stop
just waiting for your instructions
[Music]
sorry it's gonna take me a minute to
login the wrong account I'm gonna do
this this dot this dot this star this
star never forget to this start somebody
composed that song for me
getting to the place right thing I can
enable slow mode
YouTube changed so many things in
creator studio I cannot find it
where is slow mode
oh yeah the worst is over all right well
I can't figure out how to find silly
slowmode anyway so here could I do it in
the chat window itself ah manage
monterrey for a toggle timestamps no
analytics Viewer activity stream health
there's no slow mode anywhere settings
allow embedding live chat found it ok
should I should I enable it or is it
it's over now I guess alright I know
where it is now so if it happens again I
will enable slow mode slow mode would
still be appreciated alright let's add
slow mode I'm just gonna should I just
do like five seconds between messages
let's try that okay okay back to our
regularly scheduled programming okay so
the first thing that I like to do with
my console setup is to install something
called oh my zsh so first of all so
there are different shell systems that
people like to use I've been using zsh
recently and then this
oh my zsh is is essentially a theme
which sets up various defaults and
colors for knowing what github branch
you're on and things like that in
terminal so I'm gonna grab this little
command here from the oh my z SH
homepage I'm going to run it and now I
am back to what I'm used to so this just
shows me what it shows me here is a
little tilde as the prompt you'll see in
a little bit if I get into a github repo
that it will show me more information
about that and various other things okay
now the next thing that I want to do is
see if I have notes I have note
installed that seems like a recent
enough version now I'm pretty sure one
of the things I like to do is make sure
that node is installing global packages
to a directory that's part of my user
account rather than like down into the
depths of the system files of the
machine across all users and there's
actually a really nice web page that
gives you instructions for that and kind
of looking forward I just look search
fixed global permissions error NPM and
then it's taking me to this NPM j/s
documentation page and if you're
wondering what's node what are node
packages I would refer you to some of
the other videos I've done that have
gone through that but this what I like
to do is put a directory and my my use
at my user level which is what this
tilde slash means called NPM global so
let me do this then I'm going to call
NPM config set prefix so it knows about
this directory then I need to add it to
the path the the path the path is an
environment variable of your machine I
mean it's different on different
operating systems where it will look up
where executables are for any commands
you type in so I'm gonna add this as
part of my path and then this in theory
I don't know if this it shouldn't work
right because I'm using zsh so I
probably need to say source tilde dot zs
let's just see what happens like what
would this gonna do there right there's
no such file so instead I would say
source zs h zs hrc so that's the
configuration file for my zsh bash so
that should add this path export to that
permanently
that is now done and now if I do
something like say npm install - g HTTP
server which is a node package i use
quite a bit to run a web server it
should work and i can even just go to
hold on here and I don't see that NPM
global because the Mac is hiding my
hidden files but I think if I do this
there we go
shift command dot by the Soyuz phone a
Mac at least will show all the hidden
files and you can see now under NPM
global that's where the node modules are
that I've installed most notably HTTP
server so that's useful to know now the
other thing is I need to make sure do I
have get yes it knows about get if I do
like get config how do I tell it to like
tell me what's in get config
I know how to like set something show
contents of git config git config list
oh so all that's there who knows what
this is just credential helper oh my
goodness I'm gonna have to do so much
here so first of all I need to set who I
am and so to do that I can say git
config - - global username Schiffman git
config - - global how's it like email
what is it get email what is it
user dot email pretty sure this is the
one that I use and get config - task
list there we go I'm lost in vim again
all I know : QQ that's what I use also I
could say I could just look at the git
config file which is there okay so
that's done but what if I want to so one
of the things that I want to do right
now in order to do this tutorial which
I'm barely going to get to today is
clone the ml5 library repo and so I like
to use an SSH SSH key and so I'm gonna
set that up right now so I should like
make video tutorials and all these
things so I'm just gonna do it kind of
quickly so let me see what happens that
should give me an error maybe it's just
gonna work right permission denied so I
could get this to work by changing to
HTTP HTTPS oh it uses less that wasn't
vim I don't know anything
and that was that was less not vim but
so what I need to do is you know I might
need to like not show you my computer
screen for part of this but because I've
got I don't want to but this is what I
want to do so generate a new SSH key
actually kind of check I don't have any
existing ones will be impossible so this
is what I'm gonna do I'm going to
generate a key with this command for my
email this is my I'm just gonna I think
all the defaults are gonna be fine with
me what's your opinion on whether I
should use a passphrase I know I should
use a passphrase but can I get away with
no passphrase livestreaming you think
I'm gonna try this
no passphrase no passphrase oh no we
have I just like allow people to hack
things I'm not adding it to github yet
all right so I feel like does this
random art image can you engineer the
key from this image that's showing me
we run through these again
oh I'm overwriting it I did that again
but I'm overwriting it so you can't see
that art image I don't know if that's a
security issue yeah
exposed okay now I need to add the key
to my SSH agent and then I need to do
this add that your SSH private key to
the SSA Jade and store your passphrase
in the keychain okay so that's done and
now what I need to do is just add the
SSH key to my github account so this is
where I am going to unfortunately I
think log out cuz I'm not log out but
just to hide my screen and I'll show you
but what I'm gonna do is I'll just show
you what I'm gonna go to up here and
then I'm gonna go to settings so I'm
gonna I'm gonna hide my screen for a
second click on settings and then I'm
going to go to ssh and GPG keys and i'm
going so on this screen I think that
it's got my MAC addresses and various
things so I'm not going to show it
there could be oh it's only the public
key that was shown okay there could be
so I'm reading the chat I I should
figure out a way to make a tutorial with
like a dummy github account that would
be a good way of doing this but I'm
gonna press Inge new SSH key I'm calling
this coding train login and then I'm
gonna paste my key in by copying it so
this by the way is how I can actually
just say copy my public key and then I'm
gonna go back into the github interface
and paste it in and add it and I'm just
gonna show you what I snow it's still
got a like a MAC address or something
there
so I'm not gonna show it to you all
right so that should be done and I'm
closing all this if I did it correctly I
should be able to go to documents whoops
yes allow I'm going to make a directory
called ml 5 and I'm going to say git
clone and I'm going to go back to the ml
5 library
and I am receiving the ml5 library
[Music]
okay there we go so the reason why I'm
doing this is because I want to make an
example that uses the ml5 neural network
function finishing off this series this
has been the stuff that I've been
teaching this semester and if I go to
the ml5 website go to the reference and
go to neural network you'll see this is
already a part of ml5 the current
release and I have used it in examples
but Joey Lee who is one of the
contributors and has really been doing a
lot of work this year and for the past
year to manage lis the development of
the library um has been refactoring it
the neural network librarian has made
some updates and changes nothing really
changes in terms of the user facing
aspect of the library but the behind the
scenes have changed and so to make sure
everything I'm doing is working
correctly I mean there hasn't been a
release yet I want to build my own
version of the library that I can use
for the purpose of this tutorial so
you're gonna get to see a little extra
thing about how do you build the ml5
library so first step was clone the repo
that's been done now and I also want to
find out I also want to look I'm already
here I want to look at his recent pull
request which is refactor and
re-implementation of neural network
because I want to know the branch so
this is the branch that I care about
it's not an image that's so weird
just get up make that into an image or
Oh like I can copy I can copy it okay
that's what I wanted to do because so
first of all you can see why I like oh
my Z SH it's showing me that I'm in a
git repo and that's the branch that I'm
on I wanted to get check out this
particular branch because I want to
build from that brain
do I need to is it already I guess it
has everything and then now I'm going to
go to the library page and follow the
instructions for building the library
which should be under contributing and
it's pretty simple if you have
everything installed basically the first
thing that I've already done these steps
like I have node and get and all those
things installed I've cloned the repo
I'm using an SSH key and now I'm going
to change into ml v library I am there
already I'm gonna type NPM install so
this should take a little bit
this is installing all the packages and
dependencies that the library needs to
be built so Simon who's commenting on
various things that I need as part of my
workflow I'm going to do things as I
need them so I don't need a list right
now is unnecessary because when I need
live server I will install it
[Music]
okay so now that I've done this one
thing I want to do is let me just open
the source code for the library in
Visual Studio code which I've already
installed
I haven't configured there might be some
settings and things and one of the
things that I really like to be able to
do is type code from the command line oh
and it worked I guess I already
installed that at some point but I want
to say code dot because I want to open
up the ml 5 library so to do that if if
that doesn't work natively on your
computer if you're if you're using
Visual Studio code come on there we go
then what you can do is do shift command
P which brings up all of these commands
and then if I type in command install
code command in path that's what you
need to do so I really like having this
feature because I can quickly from the
command line just open a folder in
Visual Studio code which I have now just
done and what I reason why I wanted to
show you package.json whoo so I'm
definitely going to need to do change
some settings let's change the font size
to maybe better font size to 36
I'm also get packages of JSON again
that's better so what I'm looking for is
scripts these are all the things I can
run NPM run commit NPM run pre-built NPM
run and the ones that I care about
actually right now our NPM run start an
NPM run build these are just aliases to
a longer command web pack dev server web
pack so web pack is a bundling thingy
framework that does all of the work of
bundling all of the source files into
one JavaScript library file and I most
often use
p.m. I'm sorry envy a web PS npm run
start because i want to run a web server
that's constantly rebuilding the library
as I make changes but in this case I
just want the current version library so
I'm gonna run npm run build maybe I'll
make this a little smaller just so it
fits in one line alright let me go so
then when I need to figure out what
Visual Studio code settings I want I go
over to the coding garden to CJ who
keeps track of all of the extensions he
uses so let's see if I can find that
coding garden vs code extensions mmm I
think we know where this is coding
garden just go to coding our website
view the gear support
could it be gear to stream no this is
you this is super useful though ooh I
need some of this stuff okay where where
where where where where where search
filter this by weight coding guard like
like streaming goals is coded garden
look how nicely organized everything is
and there's like a custom search and a
link it's just wonderful Koni garden
does such a great job but they know
where that is maybe it's on the github
hooting Garden github let's try this
yes code settings there we go looking
for it we're at 1 hour Thank You Simon
for that reminder I am getting close to
my contractually obligated break theme I
kind of am a default theme sort of
person the thing that I want to know is
style formatting beautify okay so let's
use let's I the thing that I like the
most is style and formatting so let's
install this
yes lint I want to have this okay then
let's install beautify I don't know it's
only got a four star rating only one
hundred twelve ratings somebody telling
me to stop now before I go forward I
guess I can always uninstall it I think
this is what I was using before as well
I know there's something built in then
something that I definitely want is save
format on save code actions on save
format on save so I like to have four it
can cause you some problems but I like
to have format a file on save let's do
that I think the bracket colorizer thing
is pretty cool too I'm a little bit
afraid to install it because it's very
visually noisy but I guess I could
always disable it it's install it for
right now so this is something that like
provides a lot of visual information to
see which brackets match up and
parentheses match up with other brackets
and parentheses it's got a lot of stars
okay
okay that's good and we can see so good
is this still going oh all right so this
this works so now we can see that it
there should now be of new folder this I
I didn't show this wasn't there before
this is the version of the library that
I just built and what I'm going to do
it's because when I start making this
coding example our over an hour into
this livestream let's see can I get away
with 32 here it's gonna be posing that
example one I want to just use a locally
uploaded version of the library and so
I'm going to add the file I'm gonna drag
that here so this is how I can now make
a coding example with my temporary
custom build of the ml 5 library knowing
that by the time this video is out that
this will be in the release so I won't
be mentioning this in the edited version
I'm just showing this now this will be
closed and I will just be in sketch KS
but I should be able to see console dot
log ml 5 it's loaded the library I can
see it down here okay
any questions about what I have done so
far yeah Nathan's thing I remember you
had prettier instead of beautifier and I
think that I did a prettier but I just I
always just go with whatever CJ well one
thing that I want to check really just
as a quick technical thing is I've been
recording with a new piece of software
called V mix and I want to just see if
oh no no no don't stop I'm gonna see
sort of like a status report 202 dropped
frames in an hour and 10 minutes my
question would be is that significant I
don't know we will find out I suppose
okay well there's a lot of messages here
alright so I'm gonna take a short break
but before I take my short break I want
to take a moment to thank today's
sponsors my friends over at dashlane so
you might be wondering hey I noticed
you're logged in to the peopie web
editor you can't even see that as as
coding train and i'm gonna log out for a
second and I'm going to show you how
easy it is for me to log in just using
dashlane logs me right in so - Lea is a
password manager I cannot live without a
password manager and you're the main
benefit that you really get from it is
just like a shortcut for everything that
you wanted to of the Internet you can
fly through forms and breeze through
checkouts it lets you fill out forms
like really quickly a one-click logins
autofill for all of your payments stuff
which I use a lot - I have like too many
can't keep track of all this stuff you
know I you know you keep all of your
stuff in a Google Doc all of your like a
list of all your passwords and credit
card numbers don't do that for you as a
password manager and it's nice to know
you can have it on your phone or any
operating system and it syncs and all
that stuff you know there are other
services like keychain and Chrome but
you're really locked into a single
system so everything's trapped in one
place and you know there's other systems
that maybe aren't as reliable maybe
don't take privacy as seriously - Layton
never has access to your personal data
and won't trick you into subscribing so
you can sign up at - Lang comm slash
coding trained there's no credit card
required
it's always an ad-free product it's like
the coding trick apparently because this
is an ad you can you can go to - like
com slash coding train you're getting a
free 30-day trial if - lating premium so
all you'll be able to try out all the
features and then if you choose if you
do like it and choose to purchase it if
you use the coding train coupon at
checkout you will also get 10% off a
discount off of the - Lane premium so
thank you so much
you know there's a 40 - Lane for
supporting the channel it's I really
appreciate it
if you're if it seems that it might be
something that interests you if you
check it out through that link that'll
know that I sent you there and that will
help me out a little bit so I'm gonna
take a break you can you can take a
minute to sign up Oh before I take a
break let me try to do something so sign
up for now or just hang out listen to
some music I have something I was hoping
this thing that I have come in before
before i livestream so I could use it
dirt as an interstitial but it didn't so
I'm gonna just give me a minute here cuz
I'm gonna I'm gonna put something
different on the screen during my break
and I'm also I've lost the chat cup and
I know there's not a lot of time left
today this was a gift this a bit of a
housekeeping livestream so let me see if
I can how I can get to this it's gonna
take me a little bit everybody it's
gonna be worth it it's not worth it well
what else am I gonna do it [ __ ] amongst
yourself
sign up for deadly donate to the
prostitute foundation any of these
things
can't see my screen right
[Music]
okay everybody come to things like very
important
[Music]
hold on everybody I'm almost there
[Music]
almost there
mediasource baby
[Music]
Oh PS not read an MOV file shoot no why
no why turn that off hold on everybody
everything's gonna be okay so VLC can
play
[Music]
mp4
[Music]
why won't it play
alright I failed the LC videos let's do
this hold on
[Music]
alright never mind I have a nice little
looping animation of this train and I
was trying to get OBS to play it so that
it would be there when I take a break
instead just gonna be this back in just
a few minutes I'm gonna get some water
okay
[Music]
auto-tuned and the internet will fix
that for me
[Music]
it's coordinates unicorns and rainbows
and cupcakes what else is there yes kids
thank you very much kittens and rainbows
and cupcakes notice that look what I get
I really
[Music]
all right
[Music]
here we are again right there all right
I noticed there were maybe some issues
because the audio is coming through the
mic and also is being sent out through
the capture card and maybe there's like
a lag between them so sorry if there's
some weird music issues I would like to
accomplish something today
beyond my rambling mess and so let me
tell you what I'm hoping to accomplish
at this point by the end of tomorrow and
we'll jump right into it tomorrow this
is it's tricky because it's for me this
is the end of the NYU semester so it's
kind of like the busiest time I'm kind
of exhausted if I'm being honest and but
it's like a lot of stuff to like catch
up on and finish up for the year and I
can I miss streaming last week so I feel
like I can undo all this stuff but let
me so you might remember that I used I
made a previous series and get my water
here about this project called teachable
machine from Google Creative Lab and
when I did this series I didn't touch on
this particular pose the pose model that
you could train so let me just show you
really quickly so for example what I'm
gonna do is call this Y right now hold
on so I'm going to record for 30 seconds
and I'm gonna give myself a five-second
delay
and let me get rid of these and let's
cord oh boy I need more room
there's my why oh it's still recording
this is my why 30 seconds is a long time
I did not need to record for 30 seconds
so pose net which is the machine
learning model that is tracking all
these points clearly this is not the
optimal alright let's let's do an M and
I guess I did 30 seconds before but it's
just 2 15 seconds this is no good see
this is a C not an M the point is you
can train a machine learning model to
assign labels to various poses and the
data for those poses that I'm collecting
are the XY locations of all these key
points on a pose net skeleton and then
if I were to train this model let's see
how that goes
well it's taking a long time
oh hello everybody I did a pose project
with the snake game right but I wasn't
using so this is a really good point
Abhijit is mentioning in the chat that
what I so this is this is great so I
actually want to reference that so I did
different poses for the snake game but
it wasn't actually the data for the the
data was not the locations of the
various parts of my body but rather the
pixels of the image that it was seeing
so in that sense as soon as I would turn
the camera and have a different
background it wouldn't work anymore but
if I actually train a model with my pose
positions then that would still apply no
matter even if I like took took my
laptop to a different location with
different lighting conditions in a
different background so this is one of
the advantages of doing it this way okay
all right I'm not going to keep going
with this what I'm going to do is I'm
going to go over to ml5 Jaso what
happened to that sketch that I had open
pose an example okay
reference
oops so by the way here's something that
somebody could file as an issue for the
ml5 website or even actually fix this by
just clicking edit this document this
should be a link to this should be a
link to this actual blog post real time
human pose estimation in the browser
with tensorflow j/s and in addition to
it actually being a link for
accessibility this is a we know one of
the principles of accessibility in
particular for users of the web who who
use a screen reader is when things are
linked with generic language it's not as
helpful as if there is information about
the content in the in the text that is
linked itself so this should probably
say something like the original post set
model was ported to tensorflow j/s by
dan Ovid you know read more at and then
the name of the article real time human
posed estimation in the browser with
sensor flow Jas or something and
actually linked the name of the article
so if anybody wants to do that pull
request right now you I would do that
here live but I've already spent enough
time doing random other things you can
actually get to it from clicking edit
document okay but I'm gonna go to that
blog post I'm just gonna look at yeah
I'm just gonna remember
just looking here
always returns an array
okay
so I'm looking I wanted to find
something I just want to be able to
reference the training data kind of
assuming that it used cocoa this is more
information than we need for this first
of this video
sorry to say
oh right there's a version 2.0 actually
belief the training code is
closed-source so I don't actually know
what it was used what data was used to
train I was hoping at least would tell
me the data set somewhere here but I'm
not seeing it
Oliver is pointing it's an hour and a
half already we didn't start I know I
know this is the this is just what's
happening this week for me thank you for
your patience I'm just gonna go forward
okay
alright let me add to this example
is that how it works create capture
video create video capture I can't
remember okay it's definitely this and
then let's do
okay
this is gonna be better anyway because
the square was a problem okay oh I
should get an external webcam I have an
idea
oh you know what I will get the external
webcam for tomorrow when I come back to
finish this Wow
okay
this is what I'm looking for okay
all right here we go I'm going to make a
video tutorial about pose net and I'll
have accomplished something today hello
and welcome to another beginner's guide
to machine learning with ml 5gs video I
in this video I'm going to cover the
that was my one start over option hello
and welcome to another beginner's guide
to machine learning video tutorial in
this video I am going to cover the pre
trained model pose net and I'm going to
look at what pose net is how to use it
with the ml 5 chess library with the p5
just library and track your body in the
browser in real time that's my statement
of what I'm gonna show you
thank you for editing you I used to say
like that'll be edited out and that
would be interest in all my videos and
now no really that's better or not I
just realized that the whiteboard camera
is not on come on whiteboard camera
how does this look
oh boy this camera is like crooked right
by the way one thing I did is I
hopefully fixed the white balance of
these cameras and they are that's better
now the white balance is yeah the white
balance is fixed now so it might look
off but at least it can be color
corrected later and it won't like auto
adjust okay
model I want to look at is pose net and
with what this camera is so much lower
than what I wrote that at a very
reasonably normal height can't see it at
all so weird cameras got lower somehow
I'm not sure what oh it's pointing down
a little bit that's fine oh you could
see my almost see my legs I'll fix that
later I know it got the sniffles all of
a sudden the model I want to look at is
pose net Mathew we'll do our like
speeding up as I draw stuff I love that
I love that even if it's just like a
little bit the model that I want the
model that I want to look at is pose net
with any machine learning model the
first question we should ask is what are
the inputs and what are the outputs
and in this case
let me start the whole thing over I got
I got it again I know what I'm doing now
everybody model as I mentioned that I'm
looking at is called pose net with any
machine learning model that you use the
first question you probably want to ask
is what are the inputs and what are the
outputs and in this case the pose net
model is expecting an image as input and
then as output it is going to give you
an array of coordinates
in addition to each of these XY
coordinates it's going to give you a
confident score for each one
and what do all these XYZ and what do
all these XY coordinates and confidence
scores correspond do they correspond of
course pot my voice is cracking
I'm sniffling it's all going going
downhill fast people and what do all
these XY coordinates and confidence
scores correspond to so I'm like I
should use an external webcam you know
what I have right here a camera an
external camera should just send this in
to pose that and then track my but
whatever I don't know how to get this
into p5 yes all right and what do all
these X Y coordinates correspond to they
correspond to the key points on a pose
net skeleton now the pose net skeleton
isn't an anatomically correct skeleton
it's an arbitrary skeleton that involves
let me actually just do that again
now the pose net skeleton isn't an
anatomically correct skeleton it's an
arbitrary set of seventeen points on the
human body that it is looking for and
you can see those listed all right here
from nose all the way down to right
ankle and each one of these gets an X
location a Y location and a confidence
score to talk about this is the craziest
last remember I don't know I have some
pretty crazy live streams
when you see the full body all I'm gonna
just add something here later for this
match and I will add something
yeah I mean we can add something good
I'd be done oh come on energy people
energy now the pose net skeleton isn't
necessarily an anatomically correct
skeleton it's just an arbitrary set of
what is 17 points that you can see right
over here
from the nose all the way down to the
right ankle that it is trying to
estimate where those positions are on
the human body and give you XY
coordinates as well as how confident it
is that it's correct about those points
let me make some footage here so this
can potentially be used actually maybe I
won't do this right now
I'll do this later
[Music]
now the other question you should always
ask yourself when you're working with
the pre-trained model is who trained the
model what data that they use to train
the model and what are some of the
pitfalls that might arise based on how
the process of collecting that data
exactly I just want to like let me let
me let me say this one more time also
whenever you use a pre trained model one
of the things
life
I started by emphasizing that when you
use a pre-trained model you want to make
sure you understand what are the inputs
that are expected and what are the
outputs the other thing that you really
want to think about as you go to use the
model is how is this model trained what
data set was used to train it how is
that data collected who collected the
data and is there any reason why you
might not want to use the model based on
that data that was collected maybe the
data wasn't oh I forgot that I'm
live-streaming to the public sometimes I
just do these for I didn't really forget
so as you do these recording sessions
like this way just go I'm trying saying
the same thing over and over again I
have to like stop myself from doing this
I'm gonna move on all right well I
started by emphasizing how when you're
using a pre train model you want to look
at what the inputs are and their
corresponding outputs but another thing
you really want to think about and do
some research is how is the bottle
trained who trained the model what data
was used to train the model how is that
data collected and what are the kinds of
questions that you want to consider
around those details that might lead you
to I got this sentenced exactly the way
that I want it good acting thank you
Chris
Abhijit has to leave physics exam ghosts
go study for your fits exams take that
don't don't don't watch this nonsense of
me repeating the same sentence this is
I'm trying to get this right because
Mike this tutorial is really not about
the the law that's big the story around
data collection for machine learning but
I want to emphasize how even if you're
just kind of quickly using pose debt for
an interactive project it's important to
be thoughtful about these questions so
this will be the last time I don't think
I need to say this thing about how I
began another thing you want to consider
when using a pre trained model is to be
yeah
something else you want to consider when
using a pre-trained model is who trained
the model and what I understand how
people are writing best teacher ever
when all I'm doing like not getting
anywhere one other important question
you should ask yourself and do some
research about whenever you find
yourself using a pre trained model out
of the box something get somebody else
trained just who trained that model why
did they train that model what data was
used to train that model and how is that
data collected suppose that is a bit of
a unique animal in the pose that is a
bit of an odd case because the model
itself the trained model is open-source
you can use it you can download it
there's examples for it in tensorflow in
tensorflow Jas and ml5 jazz but the
actual code for training the model from
what I understand or what I've been able
to find is closed source so there aren't
a lot of details koko dataset
a data set that is commonly used for
training models a data set that's a data
set that's used often in training models
around images is Coco are common objects
in context and it has a lot of labeled
images of people striking poses with
their key points marked so I don't know
for a fact whether Coco was used
exclusively for training pose net
whether it was used partially or not at
all but your best bet for starting point
for finding out as much as you can about
the pose net model is to go directly to
the source the github repository for
pose net in fact there's a pose dent 2.0
coming out and there also is a wonderful
web there also is an excellent blog post
called real time human pose estimation
in the browser was tensorflow j/s that
was written by Dan Ovid that was written
by Dan Ovid yeah I'm just that was
written by dan-o fed with yeah that was
written by Dan Ovid with illustration
with thank you I read everybody's so
nice in the chat today thank you
your best bet for doing some read yeah
you should also make sure I would also
highly suggest you read the blog post
real-time human pose estimation in the
browser with tensorflow JS by Dan Ovid
and editing and illustrations from Irene
Alvarado and Alexis gallo so there's a
lot of excellent background information
about how the model was trained and and
about how the model is trained and other
relevant details
okay now
no humans of it yeah although I want to
say one other things
there we go
and if you want to learn more about the
koko image data set I highly suggest you
check out the humans of AI project by
Phillip Schmitt
if you want to learn more about the KOCO
image data set I also would point you
towards the humans of AI project by
Phillip Schmitt which is an artwork an
online exhibition that takes a critical
look at the data in that data set itself
okay
now most likely if you most likely if
you're here the reason why you might be
interested in pose that is because wow I
can make an interactive project that
just knows where my hands are my
shoulder is and and anybody can walk up
to and interact with it I could do body
tracking in real time in the browser
with a web camera and yes in fact you
can so that's what I'm gonna show you
code wise how to do right now and the
easiest way to get started with that
from my point of view is the p5 web
editor with the p5 library I have a
basic sketch that's just opening up a
connection to the web cam and drawing
the video right here and now I'm gonna
add the ml 5 library to it so I have the
mmm mmm yeah take deep breaths it helps
speak without stuttering whoo I got a
little dizzy there
now for you in many cases if you're
interested in Poe's net it's just
because wow I have an interactive
project I want to make and I can but
anyways now for you watching this video
my guess is you've arrived here because
you want to make something an
interactive project with gesture with
body tracking and pose that is a really
quick and easy way of doing so in the
browser okay
twelve o'clock okay was a little more
time
if you found your way to this video most
likely you're here because you're making
interactive media projects and PostNet
is a tool that you could use to do
real-time body tracking very quickly and
easily it's it's frankly pretty amazing
that you can do this with just a webcam
image so one way to get started which in
my view is one of the easiest way is
with the p5 web editor and the p5.js
library which very so i have a sketch
here which connects to the camera and
just draws the image in a canvas you
also need to make sure you have the ml5
library imported that would be through
you also want to make sure you have the
ml 5 J's library imported the most
recent vert you also wanna make sure you
have the most recent version of the ml 5
GS library imported and that would be
through what is it that you a script tag
that's what I'm looking for you also
want to make sure you have the ml 5 GS
library imported and that will be
through a script tag in index.html and
will will create up create something
that'll just go on the screen therefore
that once you've got all that setup
we're ready to start coding so I'm gonna
create a variable called pose net I'm
gonna say pose net equals new pose net
oh no new no none of that
I'm gonna say I'm gonna say pose net
equals ml v dot pose net so all the ml 5
functions are initialized the same way
by referencing the ml 5 library dot the
name of the function in this case pose
net now typically there's some arguments
that go here and we can look up what
those arguments are by going to the
documentation page
here we can see there's a four rut here
we can see there's a variety of
different ways to create the post here
we can see there's a variety there's
here we can see there's a few different
ways to it here we can see there's a few
different ways to call the post net
function I just want to give it a video
and the way I want to do it actually
isn't even listed here I just want to
give it a video and a callback
interesting
here we can see there's a few different
ways we can call the post net function
for us I just want to do it the simplest
way possible I want to give it the video
which tells the ml 5 pose that function
that I just just I'm just gonna keep
going give me a second I'm gonna stop
stop I'm gonna stop stopping every two
seconds I'm going to stop stopping stop
stopping just gonna move along here here
we can see there are a few different
ways to call the pose net function I
want to do it the simplest way possible
I'm just going to give it the video
element and a callback for when the
model is loaded which I don't even know
that I need
I'll make sure there's no errors and run
this again and we can see pose net is
ready so I know I've got my syntax right
I've called the post net function I've
loaded the model
hoes that works a little bit different
the way post that works it's actually a
bit different than everything else in
the ml5 library and it works based on
event handlers so I want to set up a
pose event by calling this method on on
pose I want this function to execute
whenever the pose net model detects a
pose then call this function and give me
the results of that pose whenever
I can add that right here and set up
pose net on pose and then I'm going to
give it a callback called got poses
and now presumably every single time it
detects suppose it sees me it sees my
skeleton it will log that to the console
right here
so this is working I can see it logging
it here
so now that it's running I can see the
all these objects being logged let's
look gets let's take a look now that
it's working I can see a bunch of
objects being logged let's take a look
at what's inside those objects Oh p5 web
editor
so while the p5 web betters built-in so
while the p5 web editors built in
console is really useful for basic
debugging sometimes what I want to
investigate all of the details of a
JavaScript object I want to open it so
even though the p5 web editors so the p5
web editors console is incredibly useful
for basic debugging but sometimes what I
want to investigate the depths of
everything that within a JavaScript
object I find it a little bit easier to
look at the native console itself so I'm
gonna do that right now
and so I have the chrome I have the
developer the browser developer console
open right here I'm just gonna stop the
sketch from running let me stop the
scheduler I've stopped the sketch from
running and I've zeroed in on one of
these objects that's been log to the
console and I'm looking at the built in
browser console here and we can see that
gives me an array so what's in the array
every single pose that it detects there
could be three or four people here is an
element in the array and each element of
the array is an object with multiple
properties there's a pose property and a
skeleton property let's look at the pose
property first whoops
why is it huh what
it's just my browser broken oh is it
because I stopped the sketch and it
doesn't know what it is anymore
yeah there we go
do this let me turn to us again
all the way once again from the top
dan dan dan the p5 web editor console is
super useful for basic debugging but but
sometimes when I want to look at the
depths and investigate everything that
there is - - with a particular
JavaScript object that I'm logging it's
a bit easier to go to the developer
tools of the browser itself in this case
I can get to them through a quick key
command or just in chrome View developer
JavaScript console I don't need to say
I'm not gonna include those details
sorry once again from the top it's time
pose net time on the coding train the
console in the p5 web editor is
incredibly useful and for basic
debugging but certainly in this case I
really want to look deep into the
internals of the JavaScript object that
is the pose and so for that I think the
built-in developer console in the
browser will be easier I can get to that
really easily and I'm just gonna pick
any one of these and here I've got I've
got one to look at so we can see it
right over here it's uh each one of
these things as an array with an object
in it that up yeah
could you start over I just got here
last time I swear for months yet poor me
toward you watching this I can't believe
I'm gonna say this again the p5 the p5
console is very useful for your basic
debugging in this case I really want to
like dive deep into this object that I'm
logging here the poses object so in this
case I'm gonna open up the actual
developer console of the browser I could
see a lot of stuff being logged here
very very quickly I'm gonna pick any one
of these and unfold it so i can see that
i have an array and the first element of
the array is a post there could be
multiple poses that the model is
detecting if there's more than one
person in this case there's just one and
i can look at this object it's got two
properties a pose a pose property and a
skeleton property definitely want to
come back to the skeleton property but
let's start with the pose property it
can unfold that and we can see oh my
goodness look at all this stuff in here
so first of all there's a score so i
mentioned how the that with each i
mentioned that with each one of these XY
positions of every key point there is a
confidence score there is also a
confidence score for the entire pose
itself and because the camera seemed
very little of me it's quite low just at
30% then i can actually access any one
of those key points by its name knows
left eye right eye all these all the way
down once again to right and right ankle
so let's actually draw something based
on any of those key points we'll use my
nose
so I'm gonna make the assumption that
there's always only going to be a single
person if I if there were multiple
people I'd want to do this differently
and I'm going to make if I'm gonna good
stop I'm gonna make a variable called
pose then I'm gonna say if it's found to
pose and I can check that by just
checking the length of the array if the
length of the array is zero then pose
equals poses index zero so I'm gonna
take the first pose from the array
I'm gonna take the first pose from the
array and store it into the global
variable but actually if you remember
the object in the array has two
properties pose and skeleton so it seems
there's a lot of redundant lingo here
but I must say poses index 0 dot pose
actually let me just do this
well I did
oh I see
it was fine yes
poses index 0 dot pose I this could be a
good place to use the confidence score
like only if it's like of a high
confidence actually use it but I'm not
I'm just gonna take any pose that it
gives me then in the draw function I can
draw something based on
okay
then in the draw function I can draw
something based on that pose so for
example let me give myself a red nose
so now if I run the sketch ah so I got
an error so pose could be first let me
take out the console.log this is a lot
of extra stuff for it to do ah so I got
an error so why did I get that error the
reason why I got that error is it hasn't
found to pose yet so there is no nose
for it to draw so I should always check
to make sure there is a valid pose first
if there is a valid pose then draw that
circle and there we go I now have a red
dot always following my nose
right Thank You Nathan yeah people are
pointing out Nathan is pointing out that
sometimes when I'm going into the
JavaScript console I can't find the data
anymore it's because when I stop the
sketch it does exist in memory anymore
it got cleaned up so okay all right so
now we have this
so as an exercise if you're falling
along you know pause the video right now
and then add two points for your hands
and see if you can have like dip two
different circles that are following
your hands did you get that working
you're all at it for you
whoops
I forgot what it's called
oh there's no hand wrists
it's the wrist and your right wrist left
wrist okay sorry
if you're following along if you're
following along pause the video and try
to add two more points where your hands
are now there isn't actually a hand a
key point it's a wrist key point but
that'll probably work for our purposes
I'll let you try that how did that go
okay Ivan added for you now
let's see if this works
those were bad should probably pick
shoulders
so one thing you might notice is this is
working terribly why is it working
terribly because I'm not showing it my
full body so most likely this goes back
to how the date oh how whoo this is
working terribly think I'm almost kind
of getting it right and there we go but
why is it working so poorly well first
of all I'm barely showing him only
showing it from my waist up and most
likely the model was trained on
full-body images take a minute here
now I turned the camera I should
probably
now I turned the camera to point at me
over here and I'm further away and you
can see how much more accurate this is
because it sees so much more of my body
I'm able to control where the risks are
and get pretty good accurate tracking as
I'm standing further away from the model
noting model as I'm standing further
away from the camera
we can also do so there are also some
days tricks we could try for example I
can estimate distance from the camera by
looking at that there also some other
interesting tricks we could try for
example I could estimate distance from
the camera by looking at how far apart
are the eyes
so for example here I'm storing the
right eye and left eye location in
separate variables and then calling the
p5 distance function to look at how far
apart they are and then I could just
take that distance and assign it to the
size of the nose so as I get closer the
nose gets bigger it you almost can't
tell because it's sizing relative to my
face but it gives it more of a realistic
appearance of an actual clown nose
that's attached by changing its size
according to the proportions of what
it's detecting in the face
we just look at the pose again
now forking for convenience I was
referencing each of those
you might be you might be asking
yourself well what if I want to draw all
the points all the points that it's
tracking so for convenience I was
referencing each point by name right eye
left eye nose right wrist but there's
actually a key points array that has all
17 points in it so I can use that to
just loop through everything if that's
what I want to do
so I can loop through all of the key
points and get the XY of each one
and then I can draw a green circle at
each location
what did I do wrong
shoot
oh it's got a part in a position good
oops
so that code didn't work because I
forgot that each element each key point
is more than just an XY it's got the
confidence score it's got the name of
the part and a position so I need the
key points index zeroes position X pose
key points index I dot position X dot
position dot y now I believe this will
work and I'm gonna turn this over here
I'll stand back over here and we can see
now I'm drawing even if I get my leg up
into the picture
I guess I hurt myself can I get a full
body picture people I'm gonna make this
happen
we're slack here weigh it down a little
bit and here we go
I only think I'm not seeing are my
ankles oh it's not there we go I got
kind of accurate there my pose okay
so you can see I'm getting all the
points of my body right now standing
about probably six feet away from the
camera
okay hold on everybody
where's the why is the skeleton blank
let me look at the example that drawers
a scout
oh it's just not this is awkward hold on
is it drawing the skeleton oh yeah it is
interesting how why does it only
sometimes do that that feels like a bug
to me like why not just draw all the
connected
I guess it's based on the confidence but
all right
so um er I did do this in a previous
stream let me I'll explain that in a
minute
I'm doing this again though because the
previous stream I did this is like this
one well over an hour it was using a
previous iteration of the library so I
don't have a nice succinct video
tutorial for Poe's net which is why I'm
making this okay
there's one other aspect to the data and
I haven't shown you here it
that's fine
there's one other aspect of the data
that I haven't shown you here and that's
drawing the actual skeleton in a lot of
the demos that you see there are lines
connecting the different points now you
could just build your own lookup table
of which points are connected to which
points but pose that will dynamically
sort of like make guesses as to which
points should be connected to which
points based on like how you're standing
I guess so and why why is there a
separate skeleton thing why wouldn't
that just be like a permanent like these
things are always connected like the
shoulder is always connected to the
elbow that doesn't make any sense to me
I guess right like with the Kinect you
just know which points it like why is it
not just like a permanent lookup table
why is the skeleton something dynamic
relates to the threshold doesn't have to
do with the confidence scores it must be
all right I'm just gonna say presume
somebody will correct me
there's one other aspect of this that I
haven't shown you yet so if you've seen
demos of pose net and some of the
examples the points are connected with
lines so on the one hand you could just
memorize like always draw a line between
the shoulder to the elbow and the elbow
to the wrist but pose that what I
presume is based on the confident scores
will dynamically give you back which
parts are connected to which parts and
that's in the skeleton property of the
and that's in the skeleton property of
the poses object that it returns when it
gets results
and that's in the skeleton property of
the poses and that's in the skeleton
property of the object found in the
array that was returned to us so I could
actually add a new global variable
called skeleton we've been good for
Halloween skeleton equals and let me
just stop this for a second
poses index 0 dot skeleton
hold on
it's gonna look up how's this work
for every there's a 0 and a 1 part a and
Part B okay fine I can loop over the
skeleton
why do I don't keep closing that I can't
memorize it
yeah that's what every
and skeleton is actually a
two-dimensional array because in the
second dimension it stores the to the
and skeleton is actually a
two-dimensional array because in the
second dimension it it holds the the two
locations that are connected so I can
say a equals skeleton index I index 0
and B is index 1 and then I can just
draw a line between the two of them
so I look at every skeleton point I get
the two parts Part A Part B and just
draw a line between the X's and Y's of
each of those make it a kind of thicker
line and give it a the color white and
let's see what this looks like
and there we go this I guess about you
you can edit back and forth between
these two shots since we're recording to
both and there we go this is my pose net
and we can actually like even like side
do them or something and there we go
that's the full and there we go that's
pretty much and there we go that's
pretty much everything you could do with
the ml5 pose net function right this is
what I'm gonna mention this so for you
you might try to do something like make
a googly eyes that's something I
actually did in a previous video where I
looked at an earlier version of pose net
and you could also look at some of these
other examples that demonstrate other
aspects for example you can actually
find the pose of a JPEG that you up that
you load rather than images from a
webcam but what I'm want to get to is
what is but what I'm looking to do with
this which I'm going to get to in the
next video but what I want to do which
I'm going to get to in a follow-up video
to this is not take the outputs and draw
something but rather take these outputs
and feed them as training data into ml5
neural network what if I say hey every
time I make this pose label that a Y and
every time I make this pose label that
an M a C and a you see where I'm going
could I create a pose classifier I can
use all of the X Y positions label them
and train a classifier to make guesses
as to my pose
this is very similar to what I did with
the teachable machine image classifier
the difference is with the image
classifier as soon as I show it a
different background or I've one the
difference is with the image classifier
soon as I move the camera to a different
room with different lighting and a
different background with a different
person it's not gonna be able to
recognize the pose anymore because that
was trained on the raw pixels this is
actually just trained on the relative
positions so in theory somebody around
the same size as me swapping out it
would recognize their pose and there's
actually a way that I could just
normalize all the data so that it would
work for anybody's pose potentially so
you can train your own pose classifier
that are worked generically in a lot of
different environments so if you make
something with ml5 pose net please share
it with me and so if you make something
with ml5 pose net or with pose that with
another environment please share it with
me I would love to check it out you can
find the code for everything in this
video in the link in this video's
description I'll see you in the future
oh okay all right everyone
it's 12:40 this this is it I can always
come back so tomorrow I'm gonna do the
classifier tomorrow I probably will
livestream around the same time 10 a.m.
so stay tuned for that and there you go
I'm so exhausted I'll take a few
questions if people want to ask I'm
gonna stop all the recording and up
start uploading that stuff it's actually
you know what for posterity sake I'm not
gonna stop it yet I will I will do that
when I can all right any any questions
any notes anything anybody wants to say
speak now or forever hold your peace
peace peace I'm holding my Peas hey the
last after lunch oh yeah so I did do
this previously in this for of code to
answer all MERS question in that hour of
code livestream but what what what the
ml five library didn't have was the
ability to access the key points by
their name in such an easier way entry
point into doing this that totally this
tutorial is a little bit better will I
ever do slam I mean not anytime soon but
it's certainly something that interests
me so someday maybe can you start
streaming a half an hour earlier please
I'm not so sure it's vens on my schedule
tomorrow it's the morning for me so I
need time to get in and get set up but
maybe will you make a deep learning
tutorial ask code bulletin so I would
say that I I mean far be it for me to
know exactly what it is that I'm doing
but I would say that I have a lot of
videos that touch on deep learning
concepts all throughout the channel
they're a little tricky to find but see
what you can find and feel free to reach
out on Twitter or otherwise if you're if
you're struggling to find assignments
asking about generator functions I have
a long list of things to get to and it's
I just have to try to prioritize and do
what I can the best that I can
[Music]
micki poses as it can you use the camp
thing to control a game like the Kinect
so yes this pose net is very similar in
terms of its tracking abilities to what
the Microsoft Kinect product us what's
very different is that's a depth sensor
that has a lot more information about
the scene so it's able to do probably I
would guess more accurate tracking and
in particular it gives you depth
information as well how about making a
sign language gesture classifier using a
low-res stream opposed net currency
that's a terrific job idea and I'll
point you to abhishek who created this
project making amazon alexa respond to
sign language using AI i'm not i
actually don't know if he used a post
net for this he might have actually just
used transfer learning and images but
it's a wonderful project i would
encourage you to check out related to
what you're asking about
[Music]
so why why me and ask what's the purpose
of this livestream I've really
conflicted about this because why not
just make short tutorials so this is
what I'm doing my goal if everything
goes went well today
this post that tutorial oh when Matthew
works on it as it will be fifteen
minutes or less
and I just ideally I would record the
edited stuff I don't know but people
seem to enjoy seeing the longer versions
in the live stream it's not perfect but
that's what I'm doing right now is
YouTube your main job it is not my main
job is working as a teacher at here at
ITP I forgot what I thought was the
other thing I wanted to mention if you
were in New York City the first ever ITP
show in Brooklyn is happening this
Sunday and Monday Sunday December 15th 2
to 6 p.m. Monday December 16th 4 to 8
p.m. the winter show of all this is
where I teach this is my main job and I
will be doing a livestream either Sunday
or Monday probably Monday but maybe
Sunday from the winter show
[Music]
I'm using ml5 feature extractor helper
to load and train mobile net model with
my own data set how can I save the
training data in this case nest Beshear
that's a good question I don't think ml5
has any built-in functions to save your
training data it definitely has a
function to save the model so if you
want to save the training data you might
have to do something custom for that I
have to think about that more but you
you can certainly save all the images or
you could you could pull out the
features and save the features I don't
know there'll be a great discussion on
the ml 5 non-existent forum I guess on
the github right now
[Music]
all right everyone I appreciate you I
will be back tomorrow against my better
judgment but I would do want to finish
this project hope hopefully tomorrow
will just be two hours because I have a
lot to do and take care of with the show
coming up live stream from show and the
processing fundraiser in addition to the
holiday songs so I mentioned the holiday
song thing but probably more importantly
is you know now I'm on a third or fourth
year doing this one year I did like
snowflakes I don't remember what I did
the other one year I did trees like fur
you know your standard holiday thing
that has a tree involved with it so what
could I do this time for us like a
winter themed bunch of like coding stuff
please suggest it in rainbow topics and
then what we can do is create a label
like why don't I make a label right now
there's the holiday telephone label
already I guess we could just use the
same one let's just use the same let's
use the same label so please suggest
them and I will try to like upvote you
can also up vote thing up vote them by
oh how there's a lot here okay I'll take
a look at these ooh
Northern Lights using pro noise so I'm
going to do some coding topics during
the holiday livestream please suggest
them here not here okay I did snowflakes
both years
[Music]
Nathan yeah if you can make it to New
York City Brooklyn
you can run audio for the live stream
chaos equations I've done those okay uh
procedurally generated machine learnings
to know people who look style get maybe
I could do train a runway model off
something wintery themed I'm definitely
gonna do that okay goodbye everybody see
you next time on the coding train la I'm
still here
[Music]