[Music]
I think you should hear me now have you
don't see me but you hear me I have a
new button that mutes and unmutes my
microphone and in a moment I have a
button it switches to the camera that
I'm in and this song ends in about 14
seconds 13:05 county waitress flows six
five four three two and you're on do you
see me do you hear me is it Monday no
Tuesday I don't know what day it is
I do it's Tuesday welcome to the coding
train feel like it's speaking very loud
and people in the hallway are going to
listen and hear me and I feel
embarrassed
hmm so this is unusual that I'm here on
a Tuesday I missed Friday I'm having a
hard time if I'm being totally honest
here's the good news I have I've setup
I've worked very hard with a lot of
people's help I can barely take any
credit for this but I have a new studio
that I am live broadcasting from and
soon I'm hoping to make the studio
available for other people in particular
to the NYU community because this is a
room that's at NYU where I broadcast my
other separate thing independent take
the coding trade it's very complicated
so studio is going well I have new
buttons I can mute my microphone watch
this I can I have this button that makes
my laptop disappear but look isn't that
weird look at that that is crazy but so
if I you need to like type in a secret
password look at that okay and then I
can make myself disappear
there go now if I just need to show you
the screen there we go
I feel like the focus might be off a
little bit I def I need I I'm gonna get
I'm going to get someone to come help me
with tuning the cameras a bit focus
white balance that sort of thing I know
also these lights
reflecting in my glasses I turned them a
little bit so maybe that's a little bit
better I don't know I do but so things
are good this is what I was saying
things are good in that the studio seems
to be working all that sort of stuff but
I'm having trouble getting sort of
regular schedule going so today was like
a target of opportunity I have a round
till 11:30 a little bit past 11:30 free
so I thought I would try to do a live
stream I missed last week so I'm gonna
try to do two this week I my goal is
once a week but one of the reasons why
I've actually had a little bit of
trouble live-streaming find a live
stream and this this may fill you with
joy or it might disappoint you I have no
idea but I have a new video that
hopefully will be out later this week
that I that I should that I did a coding
challenge in that was not from the
studio and it's back from a different
secret location then I will not say
anything more about when the video comes
out you will see this new secret
location I don't have the capability of
live-streaming from there so it was
recorded and then edited but I hope to
be doing these new this new style of
video in addition to the live stream so
not replacing but in addition and let's
see how that goes so I look forward to
when that comes out later this week I
don't being very vague about it I look
forward to hearing your feedback I will
provide a sneak preview to this video in
our coding train member and patron slack
group so if you're thinking about
wanting to join if you want to see a
video like 24 hours earlier than
somebody else I don't know why that's a
thing now would be the time to do so
I'm also planning to I'm looking a
little on the bright side it says alka
but i'm looking sharp
thank you see the problem is when I look
at the chat I lose my train of thought
it's a sad life it's so sad
ah I will have some new announcements -
about the rewards program for men coding
training members so I'm revamping that
I've got some new ideas that are kicking
around some new stuff will be launching
so hopefully you're hear about that
later this week are next week and just
on cue we have a new member
Aaron B so welcome Aaron beam now oh no
I haven't implemented this yet soon
there there's some new - some new tools
in the YouTube membership thing I
shouldn't go on too long about this
because most of you are just you're
watching and you don't want to hear me
talk about the membership stuff but once
you join there should be information
about how to get your slack invite and
all that sort of stuff it might not have
shown up for you but we're fixing up
that stuff and hopefully by the end of
this week that we'll be in better shape
okay so let me just first before I go
off into my rambling nonsense my goal
for today is to do two tutorials on two
functions one tutorial on each function
there's two functions one tutorial for
each that makes two in the ml5 jazz
library if you're not familiar with ml 5
Jas now is a good time for me to open up
my web browser and tell you the ml 5 GS
is a friendly machine learning library
for the web it is a library that is
built on top of tensorflow j/s it is one
that I work on with a lot of different
collaborators here at NYU and throughout
the greater open source community and so
I have a playlist going of video
tutorials there's some really exciting
what I find to be incredibly exciting
new features that are in development in
ml 5 they're not part of the release yet
if I can get my act together later this
week I'll do a second live stream with
those new features but I wanted to fill
in some gaps about a couple models that
are here in ml 5 that I have not touched
on in particular unit and body pics and
both of those are examples of image
segmentation so Ojai coding garden with
CJ
great I love this member think he's so
happy
I'm gonna get some lights that blink or
something when somebody joins that'll be
fun
shurik Schaick thank you so much when
you first join you get a little this dot
iconic what is the world what is this
world and my body is it okay that I'm
buying into this that you'd like sign up
and you for this membership thing and
you get it emoji icon okay so the
websites starting to look different
already we're doing some new stuff with
the website I'm super excited about it
but so this is that's the goal that's
really my only goal for today I want to
show community contributions I want to
do two tutorials I want to thank the
sponsor of the coding tre watches I even
have a button on my mouse buzzer oh god
oh stream deck that will bring this cup
up so thank you I have the this live
stream is once again sponsored I took a
break from that for a little bit while
it's figuring things out but I think I'm
settled so thank you to brilliant
org if you're interested in finding out
more about brilliant tonight hopefully I
opened up the correct banner I can't
remember me - duh or just coding train
hold on I might have to fix this this is
welcome to my incompetence II hold on I
know how to check this just talk amongst
yourself for a second
oh yeah new member mama who's supposed
to be a coding livestream not just to
thank the new members livestream oh
where do I go well this is very hard to
use this computer over here it's a PC
can anybody see the description of this
video I kiss oh wait I know where to
look I have it definitely have it
correct in the description that's right
up to look I don't swap it out here's
the live stream no it's me
oh yeah that's a mistake let me fix that
banner hold on
live exciting oh I thought I was being
so good and I was gonna have no issues
I don't have it on worst
[Music]
all right well I will take a break I'll
take a break and fix that later all
right back to Java so if you're
interested in learning more about
brilliant go to brilliant org slash
coding trained not Nova Nova I mean the
that is part of the coding train but
this particular URL is just slash coding
tree want womp-womp-womp okay let me
okay I'm gonna next thing the next
segment I want to do is show you
community contributions but I have
something else I have to show you first
gonna have to send an email about the
air I can feel myself palpitating new
member again this is crazy
hello Jonathan Barbra this is nuts
and I've never had so many new members
all at once so I'm about to change to a
different camera view here we go I'm
gonna press this button go over here now
I it's this camera is set with a lot of
automatic settings so I think it's gonna
start to lighten up cause it's like auto
white balancing and it also has auto
focus hello but I am very excited to
show you something this isn't it's a
smart surface it's magnetic so I can
hang things up here which I'm very
excited about that should open up a lot
of possibilities and this is a sheet of
draw a dry-erase sheet so I can now
write here and once again I have my
ability to diagram I don't know how
visible that is for you and certainly as
I come closer it's gonna go out of focus
so I probably should focus the camera
specific
they just on the wall and tune the white
balance and all that stuff but at least
for now and have this green cloth at
least for now I can use this as a
whiteboard so that's what I'll be doing
today you might remember last week it's
over here you can't really see it I
guess I could but this was my whiteboard
from last week which still has all the
stuff that I listed on it but it's just
over here in the corner right now the
thing that's great about this is it
seemed a far as I can tell it's matte
and it's not really reflecting the
lights too too much so it should this
should work better than what I had
before
alright so that's the other new thing
and then I come over here and I press
this button and then I am back over here
okay people are asking when was the last
time there was a livestream with Simon
absent you would think something must be
wrong if there's Simon's not here oh and
I don't have the Select channel open let
me get that open here we go oh my god
too many things how do I do this
there we go very faint all right so
we'll see how this goes today each day I
make each time my livestream I make a
small incremental improvement I found
this bug in the kitchen by the way and I
loved it I think this is a Scrabble mug
it's got an N with one and then all the
other letter distributions back here
okay now if you recall one of the things
that happens on the coding train is I
make a coding example like a coding
challenge and last week previously
previously on the coding train I
programmed a ukulele tuner now I should
mention and many of you rightly pointed
this out and I don't know if any of you
who are actually watching this live with
the random people who showed up it's a
one video that I made many rightly
pointed out that this is not a
particularly accurate or efficient way
to make a ukulele tuner or to accurately
estimate pitch based on audio input and
they're certainly like
highly specific mathematical ways of
doing that related to a FFT algorithms
and sound analysis more what I was
attempting to demonstrate was a
particular machine learning pitch
detection model that happens to be in ml
five so it was a nice opportunity to use
it to create this tuner so if I click
here on to the challenge we can see the
videos here and I'm gonna just quickly
run mine see if it still works it's not
very promising
oh I know what the issue is wow this is
a weird this is ah you know what this is
a good thing to bring up so why is this
not working I asked you the viewing
audience and then you will near this
question about 30 seconds from now and I
won't be able to wait till you try to
suggest an answer so the reason why this
is not working is Chrome for good reason
I would say but I think this is a
debatable point but will not begin any
audio source any playing of audio or
recording of audio and that's probably
it even more important in terms of
privacy without a user interaction first
so in other words we want the Chrome I
suppose the reasoning is to protect
against a website that might syrup tisha
sleet record your audio you can imagine
how problematic that could be so I
didn't so in this I prefer me to
actually have programmed this ukulele
tuner so that it would work we need a
button like start listening or something
which I didn't include because if I just
go to the actual editor view there is a
user interaction that happens on this
page me pressing this Run button and now
it works so while I'm in the editor view
it happens to work but when I'm not it
doesn't so I fibula tune ukulele
[Music]
okay so that was this was the project I
made I would say the big issues with
this the things that I don't like about
it are mostly all have to do with user
interaction and visual design and I
would say these are not my strengths I
don't know what my strengths are but
they're certainly not that and so what I
enjoy doing with these coding challenges
is making the sort of like technical
demonstration of the idea and asking the
viewing audience to make their own
version of it maybe with a creative
twist or being more thoughtful about the
interaction design lots of possibilities
there so I'm gonna close my version out
and I'm going to go now to this page I'm
gonna show you oh wait so many of these
came in okay well 3m Italy three of them
are from Simon but there's a lot of
these contributions so I would like to
spend more time in my live streams
sharing work that people are making from
the audience and from elsewhere in the
world you know I really want to feature
a lot of different kinds of people who
are doing stuff with code from different
and so I'm gonna take a moment to do
that right now so I'm just gonna click
through these in order and we'll sort of
see how they work and give my commentary
on them what could go wrong what could
possibly go wrong so let's try this one
by Luciano tonay allow I don't need to
translate the page okay so let's see
here I think we so I think we're gonna
run into this issue and we can look to
see the audio so this is what I'm
talking about
user gesture needed to start audio
context please click so I don't know how
to force
this to work like if I can click fast
enough oh there we go
so I got lucky so I just clicked a bunch
of time so now let's try this so I like
this I like so that one thing I like
about this is it's closer to the what
you might typically see with the tuner
where there's kind of a dial and the
dial is when it's centered is at the
right frequency I might say that this is
jumping around a lot a lot of that has
to do with pitch detection model and the
fact that I'm speaking and there's noise
in this room but some type of
interpolation some other visual
indicator right now I think this is a
problem with mine is just the like on or
off quality of these is it tuned or not
you want to be able see am I getting
closer or further I think could help
here but I like this idea of having it
be more dial like okay let's try this
one by mine bill I think I'm just gonna
click into the editor I love this I'm
like filling up a circle let's try
changing the note and see what happens
[Music]
you can see how the circles getting
smaller again we've got this a lot of
noise and this sort of green or red and
no interpolation there which I think
could could potentially help I like that
there's this option of changing it to a
guitar that is cool so the notes are
different what it's looking for
basically on zits are sometimes CJ
writes sometimes having the dev tools
open before the page loads allows the
mic to work without interaction oh and I
see Simon Tiger is just typing so I
think Simon has made it to the live
stream everybody do not do not worry
everything's okay in the world okay next
we have the ukulele tuner with a pitch
trail by original Aang let's try this
one oh oh I love this it's almost like
those you know guitar hero style games
where the notes are coming at you and
you have to time the interaction there's
this and I love the idea that I could
create a picture from playing music here
in a way I wonder if there's something
to this beyond tuning which is a way of
visualizing music and sound wonderful
idea here ukulele tuner same model new
design by David Schneider whoa whoa now
this is great embedding the tone
generator is a very smart idea and then
here we've really got a nice dial going
let's play so let's move this around
so something seems a little off with the
position of the dial again this is not
the perfect the perfect way of doing
pitch detection but I this now we're
really starting to see something that
has a very you know traditional dial
like design I like that there's a little
indicator of what the mic is doing
somehow this worked right out of the box
I wonder if if you know why if there is
some kind of like user interaction
embedded that I don't realize that I've
done here but this is great to see what
if I play different note it's Auto
detecting their note it's great great
job thank you David and now whoops we
have three submissions from Simon let's
look at them tuning all strings at once
what is that even pause that possible
[Music]
let's see if the tip yeah so you'd think
the dev tools opening would help but I'm
just going to go to here and I'm going
to do this like this
so let's strike to you a little bit I
[Music]
like the yellow and I like the sort of
like yellow color as a kind of warning
sign I'm almost there
another thing works pretty well having
trouble getting my ear isn't so great
I'm wonderful
I love I like this of having all of them
on one page here that's a nice design
idea click to tune the next string so I
think this are like different
interaction design ideas right where one
mine is just auto-detects which string
another possibility is showing all of
the strings and here's one where it's
actually giving me instructions so this
is nice it's like a tutorial walkthrough
which i think is always helpful to the
user so G is this one I suppose if I
click it saw me to do so I like this as
a design idea as well and then random
number generator mmm this I don't know
what this is let's see alright I'm
fascinated by this what is going on here
Simon
temp error pitch pitch is it generating
a random number based on the audio
that's coming in well there's four
submissions
yeah violet writes I feel like this kind
of project would be much easier in
max/msp yes so p5.js the browser is not
necessarily the optimal context for
doing you know for creating this kind of
sound processing application certainly
other programming environments are you
know geared and geared entirely around
working with audio so I think that's a
good point okay so I don't know Simon
I'd love to hear more about this random
number generator one two three I only
see three
according to Simon therefore oh wait
original ukulele tuner I somehow missed
that one okay so this is just a version
of it that's much like mine with a
slightly different interface okay great
so thank you everyone for your ukulele
tuner submissions I really enjoyed those
next segment I'm gonna look at some more
community contributions so let me go to
you know me and I'm obsessed with this
idea of spinning a wheel so let me go to
github.com slash coding train slash
website and we'll see david viewer david
has created a wheel that actually will
randomly pick a viewer contribution to
feature
so let's this is a little trial run here
of this idea maybe we're going to work
on this more so I think this is I
actually don't know I think this is
pulling from a spreadsheet of a user I
don't like the word user you know what
the word is and this is not my idea I'm
sort of pulling this from a talk or a
presentation that Dan Pfeiffer who is a
wonderful programmer and activist and
person on the Internet
people there people the people not users
they're not users to manipulate and have
make them use your stuff they're just
people so this is the wheel that it's
it's numbers so I've reduced people to
numbers which is maybe a little bit
reductive that's the word but these are
all tied to a particular person viewer
contribution and each time it's going to
pick ten random ones and once it picks
ten random ones I can now spin the wheel
and it might be nice to see what the
people's names are I can also do my
little drum roll here I don't know what
this one is number four who is number
four passengers
Thank You CJ passengers on the train you
know the Train thing was just like I
needed a name for this channel within
like 24 hours it's like somebody's like
I like that but I don't know that the
idea of the Train theme has really come
all from you know for me really this is
really a channel about coding and and
rainbows if I'm being perfectly honest I
just want to put that out there just for
a minute wouldn't put that out there
okay
back to this so I believe if I mug is
blocking this but I believe now if I
click this button remove and view
selected it's going to take me to this
so we picked
ah here we go he picked item number for
this the challenge this one is based on
is Perlin noise Luke's and this was
contributed by Maxime anakov alright
overuse this now HBU it here you can
open a new window Wow so I believe this
challenge if I remember correctly was
showing how to close a loop all the way
around a circular path with the kind of
the radius than any given angle being
tied to Perlin noise and what's a nice
application of that it looks like making
like wobbly little Wiggly asteroids
thingies in an asteroids game this is
really good look out look how it
labyrinth us is so I want to make this
easy I'm gonna give myself three lives
and I'm gonna start the more difficult
than level the smaller number of lives
the more points you get controlled by
arrows press escape to return to the
menu here we go wow this is great
sorry I have to end my livestream now
and spend the rest of the day playing
this wouldn't it be fun to work on a bot
an AI a little evolutionary algorithm to
evolve a strategy for playing this game
wonderful wonderful job thank you - I
already forgot who this was Maxum man
akov for this variation on parole annoys
lips let's look at one more let's look
at one more just I want to be
conscientious about the time I do have
some tutorials I want to do today so
let's look at one more
it's been the wheel
number three this original challenge is
blobby wow there's gonna feel life do
you have a lot of challenges based on
blob like shapes and it's created by
Grilli 86 let's take a look and see oh
there we go
so this is nice ah so this looks like a
pretty exact replica of my challenge if
I remember it correctly I don't think I
had this sort of changing color so it
looks like what's been added here is
that the color is changing according to
maybe a noise algorithm but also it
looks like if Ohno there's you know
what's nice about this is seeing it on
code pen and actually looking at the
code looks like there's a bunch of other
parameters that are changing and
happening there's like lots more going
on so this is really nice I mean this is
actually oddly one of my favorite things
to visualize it's just a kind of
undulating gelatinous like shape things
that are spongy that are soft or quite
hard to do with code and very satisfying
I think to play around with and and use
okay great job that ends that segment
you have segments now how's it going
it's going very well thank you
alright I'm gonna close these windows
out I am I am a smart surface I am a
smart surface people like me just you
know daily affirmation nothing's wrong
with little daily affirmation all right
back over here so if you're wondering
how to submit your own coding trained
community version viewer person
submitted passenger variant on a
challenge go to the website and this is
a link to add your own version and it's
a little bit tricky you have to use the
github website you have to you do
something called a pull request
but we are here we the coding trade
community are here to be friendly and
welcoming and help you through that
process you can't do it wrong you can't
break anything and even if you did break
something that would be awesome
if you can figure out how to break the
website that would be fun and a learning
experience for all of us so don't be shy
if you think that your contribution
isn't valuable you're wrong it is
valuable please submit it and I even
have a video I think that goes through
this like community contribution coding
train I just google that there we go how
to add your contribution to the coding
train website so this video hopefully
give you a little more context all right
did I plug my cat Instagram yet today
all right so here's what here's what we
got I've got a little over an hour and
I'm a three more things I am going to do
a quick tutorial on oh I'm gonna do I'm
gonna look at the you net model in ml
five let's do a daily challenge on
brilliant org and then I'm gonna look at
body pics in ml five that's what's
happening for the rest of this
livestream today so give me a second
here to get set up for the next bit
[Music]
alright by the way I've been seeing this
warning / error a lot recently and I
don't know what it is a cookie
associated with a cross-site resource at
cloud fair comm was set without the same
site attribute why is this appearing cuz
I have something to do with loading the
P 5 library from the CDN
oh it's Klis must be right because
that's cloud fair no more warning huh so
is this what is this I don't want to get
off-track here but I'm wondering is this
an issue with cloud fair with the p5 web
editor who knows right somebody can
write to me or I probably might not see
it in the chat right now but we movie
can somebody can help me with this in
slack I see Alka is typing and I'm
almost certainly certain the answer is
about to come but this is not an error
that I'm super familiar with so I want
to do this tutorial not in the p5 web
editor because I'm not super familiar
I haven't actually this is a thing that
I do which is I make tutorials and stuff
I haven't used just right off the cuff
why not but I'm not super familiar I
haven't really done a lot of work with
this unit function so I I think
debugging it or figuring out what's
going on might be a little easier in the
browser you can close the what's new
a part of the console I did that I'm not
so sure about the cookie area yeah okay
all right so one thing I can do it at a
minimum is I can tell it to not show my
warnings so I'm going to just take this
off and then we won't see the warnings
all right so what do I need I want a
couple things before I start this
tutorial
I want the ml5 website closed the web
editor I want to reference so I'm going
to spell this wrong but I'm hoping
Google will help me is Syed's github and
Twitter and website Alley FBI maybe
somebody could help me who's watching I
should have asked Sayid this am i
pronouncing this within the realm of
reasonableness for a silly American and
I am xayide
of up alley Fi Syed Ali fi and somebody
help me with this so um while I'm
waiting for maybe the chat to help me
with pronunciation Adam asked unit are
you going to do some segmentation yes so
I am here to demonstrate how to use a
pre trained image segmentation model
which has the unit architecture in the
ml5 genis library so this is not a
tutorial on the unit architecture coding
it from scratch with you know tensorflow
and c++ I'm really just using a very
high level tool to have a pre trained
model and sort of look at image
segmentation in in ml 5 GS okay
okay let me see if I can well thank you
I got some sort of super Chad thing from
Christopher Jessa thank you make me feel
very odd we're not comfortable but I get
slave I appreciate it
Oh bro train whistle oh that's very kind
of you it very much appreciated it's
more like zayeed
so did I get that right armor xayide yes
I people area what all right do coding
challenges holy Jessup I'm reading the
chat I was looking for something here I
was gonna make some people moderators
but okay I did notice it's just that it
takes me it take some time right you
what you don't understand
I mean sometimes I don't notice but you
just really have to be aware there's
quite a significant delay in what you're
seeing me versus the real time of the
chat so well anything that I respond to
in the chat HAP you won't see it for at
least 30 seconds all right okay xayide
okay
okay here we go and like join you just
cycle the cameras are they gonna shut
off
it's 10:30 I've got an hour here so I
think we're in good shape and here all
right hello and welcome to another
tutorial part of the beginner's guide to
machine learning in JavaScript with ml 5
GS
so in today's tutorial I want to talk
about a particular I think you can hear
that yes you can hear that yes I'll just
start over but I almost feel like I want
to keep that kind of sort of funny hello
and welcome to another video as part of
the beginner's guide to machine learning
in JavaScript with ml v GS so in today's
in this particular video I want to look
at something called image segmentation
I have a whiteboard I'm just looking at
how well you can see that so what do I
mean by image segmentation so right now
you're looking at an image of me and
there are different parts of the image
that we as human beings could dissect or
deconstruct so let me try to do a very
poor job of drawing a version of this
myself
and maybe there's also sort of this so I
could think of this image as having
different segments different parts
there's my head there's my body there is
the background which is maybe a wall and
then there's actually a sort of
secondary wall over here that's the wall
to my right and this is the wall to my
left I could even think of that as
another segment so in this case I have
full I'm just decided this video has
four segments a B C and D and in a way
this is ultimately a classification
problem what I'm doing is I'm looking at
every single one of these pixels in this
image and trying to classify it as one
of four possible labels and you could
imagine how this could be useful maybe
you want a computer vision Pete software
to be able to find all the pixels that
are cars on the road that are flowers in
a garden there's so many possibilities
here and so this is there there are so
many possibilities here and there are a
couple function there what am I trying
to say here looks like there's a third
segment I don't know what's going on
there are a lot of possible applications
of this so the there's a much longer
story to how you might collect a data
set that is labeled and segmented train
a model to recognize image segmentation
and maybe this is something that I can
do a deeper dive into in future videos
but in this video I want to start with a
particular segmentation model that is
trained on faces what it is yeah I think
its face and really just dip that's
trained on faces and really just
differentiates the image into one of two
categories
it's the foreground or in the background
it's the human face or it's not part of
the human face and this is a quick way
to do masking or other types of things
that you might want to do in an
interactive media application the
architecture of this particular model is
something called u net and u net I will
include a link to I will include links
to more resources about what the unit
architecture is if you want to get lower
level with this but this is a
particularly Sheen learning architecture
that was designed and invented for
medical imaging so you could imagine how
image segmentation could be useful with
medical images if you're trying to pull
out different parts of the anatomy or
recognize certain aspects of the human
body what else do I want to say about
this
wait there's another super chat this is
crazy
Christopher Jessup
oh it just says it twice hopefully
that's not a mistake turning like bright
red here alright okay if you're looking
for data sets that are available to the
public with training data for image
segmentation a very well-known one is
called cocoa or common objects in
context what's the how does that make
cocoa common objects in context wouldn't
that be C oh I see time-wise where are
we okay
common objects in context Oh
context is C oh I got it figured it out
the cocoa data set are common objects in
context can you see this it gets really
bright as I get closer here the white
balance is a problem oh well so in the
next video I'm actually going to look at
a particular image segmentation model
called body pics which you can start to
think about is how that must have some
similarity with this face unit model and
this particular model was trained off of
the cocoa dataset but let me talk to you
about the unit model that's in ml 5 the
one pre-trained model that's there and
how it was trained and who trained it
and who contributed this to ml 5 okay
Christopher says it's not a mistake
yeah thank you but and I I appreciate
that it's the last one for today okay
all right so
hold on I just want to
Sayyid the pre-trained model the
pre-trained unit model that's in ml5 for
you to use was trained by a researcher
named Syed out of all efi let me say
that one more time because i feel very
awkward about it
the pre-trained model that's available
for you to use in ml5 was trained by a
researcher names ie Syed Ali Fi and you
can find out more about Saeed Saeed the
restraint the the pre trained model
that's in ml5 that you can experiment
with was trained by researchers ie alia
fine and and Zayed you can find out more
about him from Twitter page github and
also a website and include links to all
those in the video's description I do
want to point out here that this
particular model was trained from this
data set and I haven't done I haven't
looked in depth at this particular data
set it is a face head segmentation data
set but if you look at these images you
might notice something sort of strange
about them I believe that this is a data
set that was created through 3d
simulation and I you know maybe some of
you enterprising viewers could look into
this more and and provide some
commentary in the comments I mean I can
come back in a future video and look at
this data set a bit more but there's
always a lot of complicated issues
around collecting data of people's faces
and making sure that you have a
representative sample are you collecting
those faces with people's permission in
general I might say if you're collecting
people's faces maybe you just shouldn't
be doing that but this is one approach
to doing so by I'm looking at the chat
and the super chat is going off the
rails but hearing but this is this is
one technique of doing it because also
even if you get real images you're going
to have to hand label them somehow but
if you're simulating the faces and then
creating the segmentation maps this is
the training data so this was the
training data that zayeed used to train
that model with the u net
Oh texture let me let me offer a quick
few thank-yous here oh those were all
Christopher chips okay thank you thank
you thank you
I'm your train whistle blowing person
here and then also thank you to Andrey
Kay Lopez wah thank you so much very
kind of you
this really allows me the the support
really allows me to have allocate more
time to doing this to bring more people
into the community and I hope that I'm
doing everything in the best way that I
can all right now let's look at the
reference for this unit model in ml 5
and try to build a quick example that
does the actual image segmentation
remember we're not training the model
we're not creating our own image
segmentation model which is making use
of one that somebody else trained and
applying it in the browser with the ML
5gs library so first the first thing
I'll do is I'll go here on to reference
and I'm going to look for yo sorry I'm
going to look for a unit right here and
whoa amazing so first of all a website
is totally like an active process of
changing and being updated oh wait where
am I even am I even on the website cuz I
went to here reference oh yeah it's
taking me to kid up dot I hope this is
crazy
like I think Joey Lee who's one of the
leaners of ml5 Jas it's been working on
the website quite a bit it looks
completely different than what I've used
to which I think it's fine this is
making it fun so here I'm on the website
I'm going to go down to you net looks
like there's a placeholder image right
now let's quickly go to acknowledgments
it's kind of look can I get to the old
website just for the purpose of this
video let's just see for a second I
think what if I change this to ml 5 j s
dot work force it to do that
nope no no that's PR g yeah all right
I'm just gonna I'm gonna live with this
right now and the construction noise
always the construction noise how am i
doing timewise okay not so bad
could be worse okay thank you too rough
sue Betar as well Wow
all right usage so I'm not gonna worry
about going into the reference in detail
I'm just going to
we'll cut to this when we make the
tutorial
please do a video about I don't know
what that is I'm gonna have to look up
what that is all right let me just let
me take a picture support open-source
software development there's lots of
wonderful projects out there that could
use your support as well as me thank you
first thing I want to do is navigate
over to the ml 5gs web page the
reference page for the U net model now
the you know I should be clear her unit
is unit is a machine learning
architecture there's no I should be
first thing I want to do is Maz is
navigate over to the ml 5 jazz website
and look for the unit reference page so
on the reference page also find some
information about the about the
background behind this functionality in
ml 5 as well as sort of like a
QuickStart guide for what the basics of
code works at code how the code works
and any more stuff so what will be clear
here is that you net is the machine is
them is the model architecture so on its
own it isn't anything but zayeed trained
a model based on phases so in ml 5 to
access that pre-trained modeling it's
the only one right now in the future
there might be more that you could put
in here you'll say ml 5 dot unit face
so let's actually start putting that
into our code so um this is what I have
so far I have a p5 sketch that loads the
video from the webcam you can see the
green screen here and it's displaying it
here and then I'm redrawing into the
canvas so what I also want to do is draw
the segmentation here and see and see
the original video here so I'll go back
to the code a couple things to mention
is um got to make sure you have in
addition to importing the p5 library you
have the ml 5 library important as well
and now I can go over and start to edit
sketch da yes so back to the reference
page I want to say this and most likely
let's just see if this works
I think I want to try to load the model
with preload let's see if that works
and then let's just try and set up let's
try console logging of the model to see
if it's been loaded great so this looks
promising model ready is true so again
you can load the model asynchronously
with a callback and there's a variety of
different ways of doing it but a quick
way to get started in p5 s use the
preload function to make sure the model
is loaded before it starts so that's
awesome the next thing that I want to do
probably is call I'm guessing call a
function like segment I want to say hey
I'm a machine learning model the input
is an image the output is the segmented
version of the image right here this
video of me is the input I'm gonna feed
that into the model the output is some
kind of image map of colors it's very
hard to do these tutorials and handle
the super chat at the same time but I'm
so appreciative david schneider writes
the tuner contribution worked out of the
box because i put some extra code to
make sure it would aha so there is so
I'll be curious take a look at it some
extra code to make sure that the audio
starts I know I don't have to super chat
but I wanted to thank you all right
yes Simon I did see your message on the
random number generator thank you I and
also if I miss the messages I always go
back and look at them later especially
in the slack so don't worry too much
okay hard for me to catch everything
while I'm live-streaming go back to the
reference we can see ah there is a
function just like I thought called
segment I'm going to give it the input
the video and then there's going to be a
callback when it's done processing
that's going to be in the result so I'm
going to lazily just copy all of this
and I'm going to put it here and set up
actually mmm now let's try this let's
see what happens I'm gonna put the
didn't set up I'm gonna take this got
result function and put it up here I'm
gonna add the semicolons because I
cannot live without semicolons and let's
just see if I get either an error or a
result all right let's see what happens
this is promising looks like I got
something I got a blob with this size
that's of type image I also got an image
and I also got a raw and I got a tensor
so this is what ml 5 does ml 5 anytime
you anytime you process data through one
of its machine learning models it's
going to give you all of the results
back in a variety of different formats
depending on what it is you might try to
do so here a clue to what I wanted to is
the fact that one of these is a p5 dot
image so for me I just want to like look
at the image I just want to draw it just
want to see it but there's also a blob
which is a sort of native JavaScript
data for the image there's also this raw
which is probably just the raw numbers
an array of all the numbers that define
that all the pixels that map of the
segments and then a tensor which is even
lower level again ml 5 is built on top
of tensor flow and a tensor is a
basically collection of numbers that
tensorflow j s is managing to throw out
the machine learning process so I think
if I just here say image result dot
image this is going to draw it into the
window I'm gonna comment out the draw
loop because I don't want to see that
right now and I'm going to hit refresh
so I should have seen the video down
here oh whoops I have an error image was
expecting three arguments I need to say
where to draw the image so let's just
put it at 0 0 try again
I don't see anything so why don't I see
anything I have a feeling that first of
all I only asked the segment at once and
I asked it immediately at the start of
the program I know the model was loaded
but was the video ready so most likely I
think something that's going to be
better here is for me to actually use a
callback right now with the video I'm
gonna say video ready I'm gonna add a
function called video ready and then
there that's when I will call segment so
let's see if that gives me something
more useful some waiting videos ready
well it definitely got the results so
here's my suspicion now the video is
ready but the very first frame of the
video doesn't really have anything in it
because it takes a like a bit for it to
like warm up I really have no idea I
think what could be more effective here
is for me to continuously call segments
right every time I get maybe what I
should have done to demonstrate this was
use a static image not just start with
the webcam feed but that's what I'm
doing here it's also I realized this web
this is an interesting webcam feed
because you have to remember here this
is not a typical image right what was it
trained on it was trained on for saw
simulated images of people wandering
through a 3d world and I don't think
they were all standing in front of a
green screen so will this even work with
me dimpling you would think like
segmenting in front of a green screen is
the easiest thing you don't need was she
learning for that cuz you could just
look at pixel colors that's what's
happening right now that's I'm making
this video so much to think about here
all right
I think what I can add here is that once
I've got the result I'm draw the image
let me ask the segment again so I'm just
going to copy paste this right here and
this should in essence loop it over and
over again segment the image get the
result segmented again get the results
segmented again Wow
what have I done wrong let's look at the
example whoops
let's look at the example see if this
works oh oh it's nice that there's a
link here but it doesn't actually go
there lets me try to find the example in
the web editor does anybody see you net
in here pose net Yolo all right let's
take more drastic measures oh here we go
oh no interesting all right let's just
for a quick second here let's just
replace the code with the example yeah
okay so it does work that's a good sign
what did I do wrong
oh I didn't know I did
this looks very similar I mean I'm just
drawing in a different place but
what
it's a video input the right no so um
this is a great question
TMC is asking is the video input the
right size and nl5
I should address this but ml5 is
handling that segmented image let me
just try doing this in draw I mean
oh maybe what comes out is no what did I
do wrong I'm just like looking at this
example right create capture size high
that doesn't matter I mean there's this
but this really shouldn't matter like I
don't need to store it oh no I need to
copy it it shouldn't matter like this is
exactly what I'm doing it's just drawing
it in a different place all right let's
well I'm running out of time here
[Music]
all right I'm gonna I don't see how this
matters but let's
[Music]
oh wait nobody took that out yeah
is there some kind of weird like order
of operations that like drawing inside
of this got result is such a problem
Oh weird huh so why why does drawing
it's actually not ready weird new member
Eckert
hello but I don't have draw so I can't
be drawing over it why
yeah I mean I could do I can draw in
here right like what if I say background
this is gonna flicker I'll just do
Freight I won't frame count modulus 255
Oh No
oh but there's no frame count but like I
can draw in there weird I honestly don't
understand but interestingly enough it's
like the image isn't ready or something
draw and draw and draw I mean drawing
and draw is a better better practice
anyway so I think I'm just gonna have to
cop to that let me go back to me not
having it working
all right this is where it wasn't
working oh whoa oh because I have this
all right I'm just gonna go back to here
this is a bit odd to me but it seems
like I think what's going on here is
drawing inside of this got result
function is problematic ultimately if I
want to draw into the canvas in an
animation using p5 s draw loop is the
place to do it I should be able to do it
here I mean there's something with the
image isn't ready to be drawn I really
don't know what's going on if I'm being
perfectly honest maybe somebody will
reveal it in the comments but looking at
the the way the actual ml5 example is
written yeah I think a more thoughtful
way if I create a separate variable
that's like a segmented image and I just
start at the beginning as a blank image
so let me just make a blank image that's
the same size of the window and then
what I can do here instead of trying to
draw the image here I can just set that
particular variable segmented image
equal to result on image and then what I
can do is I can add the draw function
and in draw just draw that segmented
image at 0 0 I mean it should already be
the correct size of the canvas so let's
see what happens now I'm gonna hit
refresh there we go ah interesting so
look so it's there it looks different
though right it's a different size than
the original image so first of all a
machine learning model has to have a
fixed size set of inputs and a fixed
size set of outputs not I mean there's
exceptions to this but certainly in the
case of a kind of traditional model
that's expecting images it's expecting
images of a certain resolution which is
probably not 320 by 240 so ml5 is
adjusting the resolution of the image
before it feeds it into this unit face
model and then when it comes out though
it comes out in the sort of default
resolution but I can just if I want it
to match up visually I can just rescale
it when I draw it like this and now
we're seeing that I have the original
video here and then the segmented image
here so this doesn't look too thrilling
because I have a green screen behind me
but let's see if I can move the webcam
to somewhere else in this room and see
what happens all right you can see
actually actually I don't know if I want
to you can see this is my like streaming
wherever there now it's starting to like
lose track of what's going on but here
it's still working now as I get over
here it thinks that's part of my face so
again what was the training data what
what kind of environments was it was it
trained in how well is it doing that
segmentation but here you can do some
basic background removal let's just see
let's just very quickly see if I can
replace replace my background using a
unit model as a starting point so how
would I do that like for example if I
say this the the pixels are actually
black and so how do I make that an alpha
Channel I could process the image right
it's probably in the raw data there's a
what did we say was in the this thing
there's the blob there's the image
there's the raw which is all the pixel
information so I could probably write ah
so I could see I mean that's
wherever there's black I could replace I
can make that the Alpha so one way of
doing that would be to make the Alpha
Channel of the image but they're like
I'm there's a mask function in p5
I'll just just bear with me for a second
can i well maybe I don't want to go down
this road additive blending would ignore
black yeah I think I'm I think I'm kind
of brown a time here I want to do body
pics and I want to do the brilliant
challenge I do have I'm just trying to
think I know I'm trying to think there's
a smarter way to do this and let's look
at the let's look at them reference
there's a mask function I thought yeah a
photo mask mask image so I should be
able to use it as a mask I'm just
curious like what if I create just give
me a second here
let's draw
just can't cure us in here
so that's good and then
wait
I will explain what I'm doing in a
second
okay that's a good sign oh no it's not a
good sign
photo mask source image last party
mission is playing by loading in late Oh
it uses the Alpha channel as an alpha
channel for this image okay I'm gonna
have to process I'm gonna have to
process the pixels I'm gonna have to
process the pixels I mean I think maybe
I don't want to get into this right now
because I'm gonna move on but I think
what I could do here is I could I maybe
I'll come back in a separate video and
do something about that or think about
how that would be useful it really
should that really should be I think it
should probably give you the image not
with I think ml 5 should practically
probably change this it should probably
put set the Alpha value to zero so Leben
I'm actually gonna do this let me do
this real quick so let me show you what
I mean
let me just go back to what I had before
and then get rid of this other stuff
one of the things you first might want
to do with this I think I just said this
where one of the things you first might
want to do with this is then actually
like replace the background behind you
so for example I could load a nice beach
scene or something but just to simulate
that let me put blue behind me so in
other words if I said background 0 0 0
255
I should see myself right now in front
of this blue background but I don't so
what ml5 has actually returned to me is
an image segmented but each the the
background pixels are colored black
rather than having their transparency
for example set to zero so I'm not sure
maybe this should be something that ml 5
actually offers and maybe by the time
you watch this this will have changed
but for right now I could at least
iterate over the pixels and anytime
there's a pixel that with color 0 0 0 I
could set its alpha down to 0 and there
might be some fancier way to do this
with like a blend mode or something
but let me just see if I can process the
pixels this way I'm gonna do this
quickly right here what I'm going to do
I'm gonna look right here where I am
receiving that segmented image I am
going to loop over and I'm gonna do a
nested loop I think actually no I don't
need to all right I'm gonna look right
here where I'm receiving that segmented
image and I'm going to iterate all over
all the pixels so I'm gonna use a for
loop to go over segmented image pixels
length I plus plus oh I'm gonna say I
plus equals four because in p5.js the
array of pixels there are four spots in
the array for every single pixel are G B
a so I can say the r-value is segmented
image pixels index I I'm gonna use a
plus a zero here you'll see why in a
second that's the red value that's the G
the G value is plus one the B value is
+2 they're all right next to each other
RGB then the alpha value is what I want
to set so I'm gonna say if R equals zero
and G equals zero can I do this and B
equals zero so this is gonna let me do
that then I want to say images pixels I
plus three I'm just gonna set the Alpha
down to zero and I should probably say
segmented image load pixels seg minted
image update pixels and let's see what
happens I'm gonna hit refresh I have the
blue background I'm gonna see myself
oh no J pixels
this is gonna make the console go crazy
just one whoops that's weird
Oh Oh not I yeah no I let me get rid of
this console.log
it's only getting that for the zero
pixel all the other pixels are
and this is an awful thing I'm about to
do because it's gonna spit out a zillion
things but let's just see what happens
it's only oh look at this this is like a
bug I'm like discovering a really weird
bug in ml5 or something which is that
the image is not available yet it
doesn't have any pixels like that's why
I couldn't draw it here it's like not
actually ready this is so weird this is
what happens on the coding train I think
I'm gonna do this like very simple
tutorial and it all goes wrong but watch
this I have a feeling now I might have
to come back and I'll redo this or like
fill in the gap on Thursday this week if
I do it like I've been feeling if I do
it here in draw boy see there we go now
I've already done a horrible thing I've
killed he'll stop go away browser
I did what I didn't want to do which is
I put a for loop in the I put a console
lock inside my pixel for loop which is
like the worst thing ever just please
stop get to the end get to the end
clothes clothes can I oh there's like a
task manager right
task manager can I get which tab is this
it's got to be this one end process
there we go
no it's still printing oh I didn't
commented out task manager there we go
and
friends don't let friends put a
console.log in a for-loop of pixels
there we go
weird so unbelievably weird okay I might
have to like figure out why this is
happening but wor this tutorial is just
gonna be you know what I'm gonna do I'm
just gonna do this again but I'll just
do it in draw and I will ignore I'm
gonna do this little little bit again
everybody please bear with me as you
always do
one of the first things you're probably
thinking of to do is oh I could now
replace the background to something else
I could put myself in a lovely beach
scene over the garden of flowers behind
me instead of just this green wall or
whatever happens to be the actual
background so the to do that you might
think okay let me let me just let me
just replace it with blue just as a test
I'm gonna say background 0 0 255 I'm
gonna hit refresh
this looks good I've got blue I've got
the image and now up it's still black
pixels not blue so the kind of image
that's actually being returned into that
segmented image doesn't have any
transparency to it the pixels that are
part of the background are classified as
background as essentially and then given
the color of RGB 0 0 0 so maybe there's
a different way that ml5 maybe there's a
way I could use it as an alpha mask or
some kind of blend mode I leave that to
the comments to give me some excellent
suggestions about it think of it this is
a better way but what I'm gonna do right
now is just manually process the pixels
so for example here and draw I can just
say segmented image load pixels then I'm
gonna also say segmented image update
pixels and what I'll do is I'm going to
do a for loop I'm gonna start at 0 I'm
gonna go all the way to a segmented
image pixels dot length and then what do
I want to do I'm gonna look at the red
value which is segmented pixels index I
uh wait so one thing I forgot is in P
5gs or in JavaScript if you're looking
at an image the pixel array has four
spots for every pixel red green blue
alpha in that order so red is pixels
plus zero and this should actually go up
by four for every pixel gee red green
blue plus one for green plus two for
blue and then when I want to set the
alpha I can say if R equals zero and G
is zero and B is zero this is probably a
more elegant way of doing that I can say
I can set the Alpha value to zero
so let's see what happens now if I hit
refresh I have the blue background I
have the segmented image and I have
there we go me in front of a blue
background you know I could do some
other things here like for example if I
wanted to just make this more of a
silhouette now I have the opportunity of
saying something like I could actually
replace all of these do this quickly
something like this and now I'm going to
see myself here and just kind of a
silhouette version of myself up here and
there's some flickering going on again I
I think I've got more to think about
this maybe I need to come back with a
follow-up video there's some weirdness
going on with the fact that I'm getting
the segmented image in this got result
callback and then manipulating it and
drawing it in the draw loop and the
order of how those things happen could
call it appears to be maybe causing some
issues here so thank you for watching
this gif give Z you net a pre-trained
face model a try make something with it
and be on the lookout for app in the
comment or a follow-up video that maybe
irons out some of these like little
issues with cleaning up this order of
operations in this program okay thanks
for watching and I'll see you soon good
bye yes so copper Francis asking does
this work without a green background
this is pretty unfortunate that I have
doing this with a green background
because it gives the illusion of doing
some kind of chroma keying effect which
I could easily do just with pixel
manipulation but yeah alright so choo
choo to the chat I'm gonna close some
stuff out here and I want to say thank
you to the coding trains livestream
sponsor brilliant org I this
unfortunately it's like fired here
instantly but I'll put this in the chat
you should go to brilliant dot board
slash coding train no the okay so I'm
gonna take that off the screen since
it's incorrect and I am going to go talk
to you about brilliant org so I
mentioned earlier that one of the things
that I'm doing now is I'm doing coding
challenges from a new secret location
soon to be launched on the coding train
Oh video coming out hopefully later this
week preview to members and I one of the
ways that I come up with ideas and keep
my mind fresh and continue learning for
new coding challenges that I want to
work on or things that I want to teach
in my classes is by the internet and I
read lots of different articles and
books and things but one of the websites
that I've been using recently is
brilliant org
thanks to their sponsorship of the
coding train and brilliant org is a
lifelong learning website where you can
find daily challenges you can find full
courses which are pretty cool I've done
some segments where I looked at some of
these courses I want to do a deeper dive
into maybe the artificial neural
networks course which has a lot of
content aligned with things that I'm
doing here on this channel but you can
get started for free just by going to
brilliant org slash coding train you can
also unlock all the features of
brilliant
org all of the different courses and
things through a premium subscription
and the first 200 people to sign up get
a 20 percent discount
again that's bleeped org slash coding
train no though maybe this also works
I'm sure they can fix that
but it's without the vine I will fix
that the next time I put this banner up
so let's go do what I actually looked
earlier this morning I didn't quickly
and I noticed this arithmetic with waves
and I thought this could be a fun
challenge for us to do together here and
me I am so me as is giving me a message
in the chat about this were like load
pixels behavior I'm gonna come back and
look at that in a second so um this
arithmetic with waves is it looks like a
fun one to try so let's look at this so
here's the question so first of all I
have if you want to know more about wave
patterns and oscillation with code
you could go to my nature of code book
if I go here under oscillation and
search for wave you can see that I
already have some code examples that
create patterns like this so this
probably looks very similar to what
you're seeing right here undulating sine
wave and one of the things I also
demonstrate in the book and I could come
back and I think I've done videos on
this is what happens when you add waves
together and if I keep going down there
we go
so more complex waves can be produced by
the values of multiple waves together so
this is just summing different waveforms
to get this pattern that has a more
chaotic or random feel to it it's not at
all it's a periodic repeating pattern
but by summing the different ways you
can get something that as a this kind of
quality to it so if we look here let's
see if we can if we can figure this out
so this is the pattern which pair of
wave functions can be added to get the
graph above and B a and C or B and C I
would suspect not a and B right this
definitely looks to me like there e are
two like the actually the these a and B
have the same period right period our
frequency is the number of time period
being the amount of time for one full
cycle frequency being the amount of
cycles per unit of time
amplitude it looks like they all have
the same amplitude being the height of
the wave so oh and Simon's telling me
there's a second course on neural
networks coming soon on brilliant Simon
who I know it's been should really be
here talking about brilliant cuz I know
the Simon makes a lot of excellent use
of the brilliant website alright so
looking at these two I wonder if I could
quickly modify my nature of code example
try that I just wanted to see if we can
get it intuitive sense you know what I
have an idea
I have an idea here which is that this
does cross zero so if I'm adding waves
together and and the waves added
together get a zero the sum is a zero I
either have to have that part of the
wave be above and below the line at the
same height right I have to add a plus
two and a negative 2 to get well not
with a sine 0.5 and a negative 0.5 or
both waves have to be zero so if this is
kind of where the zero is like anywhere
if I pick here and I go down so this a
is not zero there Oh be a zero there Oh
see okay it's got to be B and C I think
that's like wick way of doing it
it's got to be B and C so let me just
give it a minute if anybody easy just
look at the center point that's another
good way of doing it the center point is
kind of like halfway to the amplitude
that the bottom limit so a is 0 B is at
the bottom and C is there so it could be
it could be a and B that tells us it
could be a and B or B and C that doesn't
give us the answer but I believe it has
to be B and C let's just see if I'm
right any other suggestions from the
chat everybody's saying B and C
everybody in the chat this is like a
phone a friend this is a nice by the way
here's another way you can do your
challenges on brilliant down org just
live stream and have people watching it
let me click B and C let me hit submit
and correct I got a one day streak
really do these every day not just a day
that I'm live streaming somebody tweeted
me tomorrow or write a youtube comment
or something I'm Way behind on reviewing
YouTube comments one of the things
actually that YouTube has now is you can
filter the comments by members so I can
see any members that commented so that I
look at it that way but also by
subscribers so I can see who's
subscribed so if you want me to look at
you I don't want to be pitching for me
to subscribe if you like the content and
want to get notifications or don't
subscribe it's totally fine I will I
actually look at all the comments of the
best that I can never mind
scratch all that alright let's see let
me see if I can get a nice simulation of
this
I think if I go to editor p5 just saw
toward nature of code sketches am I
gonna be able let's look for a wave wave
ABC additive wave so this is additive
way this so this is the example let's
say what I don't let's let's like cancel
the cancel the increment incrementation
of the wave there we go but it's doing
it randomly so but anyway you get the
idea
oh how can I make a slack why should I
be using discord instead of slack what
let's discuss this at some point
it's 11:30 I'm not gonna get to body
picks today it doesn't seem and if I
could do that really quickly with all
that I'm sure gonna stuff I did with you
net might allow me to just quickly in
like 15 minutes make a body picks
tutorial s repudiation rights I'm I'll
be applying to NYU this application
season I was going to study straight
math but your course of inspire me to
branch out find practical ways to
express my mathematics that's lovely
yeah so I encourage somebody wants to
try to make a version of this I Kurt so
this way I'm gonna take a short break
I'm gonna try to do it look at body pics
at least really quickly I'm gonna take a
short break I'm gonna encourage you all
to go sign up if you haven't already at
brilliant org slash coding train no the
and and while you do that um see if you
can get my additive wave code or see if
you can write a quick p5 sketch to to to
create these waves and add them together
and see if you can like prove this
result oh let's see
staff right okay so this looks like is
where the graphs have 0
oh yes actually you could look at the
center because yeah oh and look at it
here yeah so this looks like it's the
same solution that I did and then oh
this is nice this is what I wanted to
create in p5.js oh you could create that
yeah so this is Alka what you're saying
to me is like music to my ears Bing Bing
Bing Bing if YouTube membership and
discord integrate I should really move
from slack I think but let's go back to
that and it's just there's lots of
excellence Jeff so you you you write a
comment here with you you got got five
minutes to do this I'm gonna go to the
wallpaper actually I'm just gonna
actually no I'm gonna leave this here
I'm going to leave this here I'm going
to remove myself how do I do this me I'm
no longer there I'm gonna mute my
microphone I'm gonna play you some music
this song is about five minutes long
I'll be back in five minutes if you can
create a p5 sketch that that solves this
our processing sketch share it in the
YouTube chat or share it on brilliant
and I will I will take a look at it
[Music]
I always forget the hi everybody I'm not
back yet but if you give me like an
extra two or three minutes then I think
I can try to do the body pics thing it's
still gonna be longer than five minutes
but I'll be back in just a few minutes
okay this this dog this dog this start
song never forget the Vista there's
somebody composed that song for me
[Music]
say once again
[Music]
and auto-tune and the internet will fix
that
[Music]
all right I am back I don't have a lot
of time a little overtime but I saw that
there were some nice comments about this
particular brilliant daily challenge
where I believe this is I have to scroll
back during the chat but someone I'm
gonna try to get the right credit here
there's so many chat messages oh my
goodness
too many chat messages I can't find it
someone will help me with this
but this figured out that this looks
like sine of 2x and this is a negative
cosine of X so if you by knowing what
the actual functions are that are
drawing these graphs you could then
visualize those add those two things
together actually we could make this
happen super fast I know I said I was
gonna do body pics but I just got a let
me let me know come back come come back
to me here okay yes leave this page
leave this page wave I just want to look
at the simple wave exercise graph wave
and wave this one right this is okay
okay stop stop stop
so let's let me I think this is
something I can adjust so let's look at
this start angle forget about the start
angle angle is just always going to be
zero and I want to draw map sine of the
angle angle velocity oh I've done some
ridiculous thing I'm going to simplify
this angle is just going to be angle
equals map X which goes between zero and
width to between zero and 2pi times four
so I want to get four cycles through the
wave um and that's I'm gonna need
something wider can I get some more
space here let's make this let's give me
let's give me some more space here why
is this actually oh I can I know I'm
standing in front of the code let me
just remove myself for a second that's
good let's make it much less high let's
say and let's actually have this go
between 50 and 150 there we go that
looks more like what we're looking for
and let's say begin shape and I can now
I can bring myself back into this
picture begin shape and shape stroke 255
no fill we can say sign of what am I
looking to do right vertex get rid of
this stroke there we go all right so
there and let's give ourselves a lot
more points to work with there we go so
this is sine of X basically sine of X so
if I were to do this
here's sine of X when we said cosine of
X then sine of two times angle so these
are both of the graph those are both and
then let's do one that's those two
things added together and cosine and
again this is like a sort of awful way
that I'm doing this but I think it's
going to demonstrate the idea let's run
this and let's comment out the first two
see does that look like the picture that
is in the brilliant challenge it does it
kind of looks like it's in Reverse right
no I'm just yeah I wonder if oh because
it's because the y-axis is flipped in a
computer graphics system so I should
really make all of these I should really
just make this one two negative one to
invert that and there we go so we have
solved this particular brilliant
challenge I am going to save this I like
to go here I don't know if anybody
posted an answer let's let's right here
is p5.js code to demonstrate demonstrate
adding two waves together let's put in a
link source code let's do this let's do
a quick
take a screenshot of this and added it
this is fun okay here we go come on drag
and drop people oh oh yeah that's what
I'm doing
oops there we go alright so if you've
made something please you comment here
or all that stuff and thank you so much
again to brilliant artwork for the
sponsorship of the coding trade now very
little time left
but with the little time I have left I
am going to go to reference body pics oh
my goodness oh look at this there's a
lot more to body pics you know I'm gonna
come back and do a video on body pics
next time let's just test out the idea
I'm just gonna do this with you in the
live context and not worry about the
sort of like published tutorial because
I do want to spend some more time
talking about body pics and I think if I
look up we're gonna see this is an
excellent poster reference by dan-o vet
who is a graduate student researcher at
ITP and there are other other people
credited here who have helped with this
particular post but this is a post that
describes the body pics model which does
something quite similar to what the unit
face model does but gives you a map of
the body with colors assigned to
different parts of the body so let's
just see if I could get this to work if
i duplicate my unit and all kind of like
this is gonna just be practice maybe I
will later this week pick back up with
this I definitely I have to you know I
said 11:30 because I knew I had to leave
by 12:00 so let's see what I get here
[Music]
let's see here if I can let's load up
this so this is a unit I'm just curious
what happens if I literally all-out I do
is just change this to ml5 body pics I
think I there's a bug where body pics
doesn't work with preload but let's
confirm that so body pics model ready so
in theory I should be able to just do
body pics here and everything should be
the same right it's the same exact
algorithm for segmenting images I'm
going to take out this pixel processing
thing and otherwise everything should be
the same that's body pics body pics
let's just see body pics that segment is
not a function
ml5 body pixel I think it doesn't work
with preload let's comment it out this
really shouldn't be up here I mean the
order doesn't actually matter for how it
works but this is very confusing to me
so let's say body pics equals ml v body
pics and then I'm going to have model
ready and function model when the model
has loaded let's try calling segment
here
and let's say the model is ready let's
see what happens here if you want to
classify a video past the video element
into the constructor
let's try sequencing this when the video
is ready let's load the model and then
when the model is ready let's call
segment let's look at them I'm glad I'm
looking at this now so I see it wants me
this is very this is revealing a problem
which is that we're inconsistent it
seems this wants me to look to actually
include the video to sort of assign body
pics to that video which probably has
some advantages but shouldn't be
required but that's probably it's
probably only going to work that way
then I would do this without passing it
the video specifically whoops
okay image was expecting p5 image what
failed let's look at the results it's
not even draw anything so this isn't
round we're two sort of debugging this
is good for me okay ah so this is
getting more stuff I'm getting masked
background mask person oh there is so
the properties that come back actually
have different names which is fine mask
background mask person so this actually
makes more sense you know in some ways
in that there are these masks that are
coming back so let's take a look at what
these look like what was one mask
background mask background let's try
that
interesting so you can see it's giving
me a solid color background remember
blue is what I'm drawing behind so
there's an alpha mask where it detects
me as the person and it's it's all it's
giving it all as one color so if I
change masks this to mask person I miss
I missus I was expecting different
colors did I not hit save hmm so how
does this particular example how do I
get this is what I want do I have to
give it the palette like if I don't give
it the palette so I'm gonna investigate
this further I'm hoping to come back
Thursday morning this week Friday
unfortunately does it seem like it's a
good day for me this semester but I'll
come back Thursday or Friday to continue
this but let me let's just try to get a
little bit further just to see and maybe
those of you watching can investigate
this and file some issues on ml5 so that
we can kind of work on this so where do
I create them all we've do this so I'm
just gonna copy all of these options
over when I say let options equal and
then I'm gonna put in the options here
let's see what happens
ask background that's mask person is it
only in the raw data is that where the I
have to like get all the colors from the
raw edge something is definitely I'm
either missing something or mask
background let's try that again
these also seem like the same thing
right there's no difference here
oh no that's inverted huh
let's look at ml5 examples and see if oh
sorry ml five medical examination is not
what I want
let's go here p5.js body pics do we see
it body pics image what Oh body pics
parts that's what I was missing
yeah look something is different in them
version this is the this is the develop
this is the release branch yeah that's
the release branch
Oh what am I doing wrong the stuff just
worked like I'm like writing the
examples in a outputs tried segmentation
threshold body pics video create HSB
palette okay that's a function video
segmentation image so there's a segment
with parts function so that's my problem
and then this is like making a palette
but I don't that I can just use the
default palette or set the colors so let
me go back oh my god we go back to my
example I didn't realize it was a
separate I'm you know read the read the
manual I think is probably a video model
segment segment with parts yes yes
segments with parts I'm tired it's been
two hours let me run this now and now
you see this this is this was my version
of it and now there's actually an image
so this is image is what comes through
here Wow mask back we got a we
definitely I definitely want to make the
AP is with you net and body pics more
consistent so I don't want to open again
of issue to discuss the naming and sort
of style inconsistencies between these
two models that would be super helpful
and now I can say result image because I
from you net I think you want the mask
right and and what just happened here
cannot read with a property undefined so
okay let me get rid of these options for
a second just use the defaults
so I create the video when the video is
ready I create the body pics model I
load the body pics model tie it to the
video and the model is ready I call
segment with parts should then I get the
results I set that to a variable call it
again and then draw that segmented image
and it would be actually interesting to
draw it on top of the video just sort of
curious what happens there and let's see
what happens there we go and now if what
if I wanted to what I could do is I
could also get the mask could do some
kind of masking but maybe if I just
actually give it a little alpha right
now I can sort of see ooh I need to say
no tint here this is what I'm going for
I can kind of see and I want to make it
much more clear there so now we can sort
of see what body picks is doing and in
theory if I could get myself further
from the camera you could see maybe more
body parts great all right so hopefully
this was helpful to you this is me sort
of like figuring this out I'm gonna come
back on Thursday and maybe do any
addendums to the content the video
tutorial that I hope to have edited
together about the unit model as well as
cover this body pics model a bit more so
thank you everybody for watching
I hope you enjoyed this livestream I
sort of hit a wall like about five
minutes ago I feel like maybe looks like
a blood sugar thing going on I need to
go eat some
so like I think I got something in my
eye but I'm gonna go I'm gonna take a
break I've got a lot to to do for the
rest of the day and tomorrow and I will
hopefully be back in McKenney a hundred
percent promises but I'm hoping to be
back either Thursday or Friday this week
if not next week and be on the lookout I
know I keep teasing this maybe nobody's
gonna like it I have no idea but I
recorded a video from a different
location not in this room and we I'm not
gonna say anything more about it
I look forward to seeing you next time
on the coding train let's see if I can
answer a couple questions
why not make a logarithmic frequency
analyzer on p5.js asks Andre yes great
question that I would encourage you to
do so that would probably that might be
a more accurate and effective way of
actually making a tuner I was just kind
of curious to go through the different
pre train models that are in ml5 library
the live stream it's always at this time
asks Felipe I wish so there was a brief
moment in time last year where I had a
consistent every Wednesday at 10:00 a.m.
live stream 10:00 a.m. Eastern I'm in
the Eastern time zone I'm in New York
City I'm in Brooklyn actually Brooklyn
New York but I haven't figured that out
yet for this year so soon I hope to
establish that but at the moment your
best way to find out when the next live
stream is or fortunately subscribe to
the channel and and click the
notification bell thing because when I
schedule it you'll get a notification
but I do expect I can tell you this week
that if there is another live stream
this week the only two times I'm
available to do so right now are
Thursday morning around 9:00 10:00 a.m.
Eastern or Friday morning around 9:00 or
10:00 a.m. Eastern so one might be
scheduled for those two times I would
probably like add it the night before or
the morning of what's the fastest
JavaScript engine on the Mac no idea
there's somebody else left answer make a
tutorial how to make a bar graph
spectrum with logarithmic frequency
scaling p5 sound I'm pretty sure I have
that p5 sound FFT I'm this video right
here on the coding train
17:11 I'm pretty sure somewhere in this
video I do something like that here yeah
there you go so I think if you check out
this video where I am very red or am I
just very pale here I'm not sure you'll
find that
[Music]
rendering 3d ray marching yeah I have I
have a lot of coding challenges that I
want to have a long list
marching squares is kind of on the top
of that list ah oh this is so important
that you're asking so remember I spun
the wheel I'm not spinning the wheel
that still got to work out this wheel
system nekima asked will you do the coal
Lots conjecture coding challenge like
the wheel said last Friday here's the
thing I already did it this other video
series so it's gonna come out soon the
first one this week is not the Collatz
conjecture I don't want to spoil too
much but I did it it will be coming out
if I had to guess right now that's one's
probably gonna be next week so I'm
hoping to have a coding challenge once a
week or once every two weeks with a
slightly different format we've ever
have Joshua Davis as a guest great idea
for a guest right now I'm not up and
running to have guests when I can get
back up and running to have guests
Joshua Davis would be a great guest
heartbeats and sound get octave bands
and log averages yeah those are all
great ideas limited time in the world oh
wait alka made a brilliant thing
let's go code dot code code pen dot io /
alka am I gonna find it here like recent
showcase all right I'll just gonna
lazily type in pen /qb be easy RW
everyone should check out coding trained
viewer alkis
code pen there's lots of great examples
there qbb capital e there we go oh this
is much nicer
much better thank you thank you for
doing that
oh I I'm no longer in my goodbye rainbow
song we'll just speed over to the end
here so thank you everybody I will leave
the wallpaper slide up for a few minutes
so the chat can keep going but I'm going
to say goodbye now I'm gonna thank you
especially to thank you to everyone who
supports me or even just watches or like
to the person who said hello to me on
4th Avenue in Brooklyn yesterday I doubt
you're watching but somebody came up to
me and say hey coding trained and I said
that's me and that was like a wonderful
part of my day so Christopher Jessup for
your super chats all the other super
chats the members the viewers the people
the people who help operate the New York
City Transit System that got me here
this morning I want to thank everyone
goodbye
[Music]