[Applause]
thank you very much thank you for having
me here in this beautiful city my talk
is called take on me take on me is
actually a 1980s song very catchy song
very catchy music with some amazing
animations it's by a Norwegian man
called aha
the thing is I'm not really good at
music I'm not going to talk about the
catchy music but I'm going to talk about
the war animations because that's like
my life go I really enjoy making this
kind of completely useless animations
it's like hey I could grab Harry Potter
and use CSS gradients and there's
something like that because if he turned
white wouldn't you so high or oh yeah
I'm ever my friend on developer birthday
protector feels less creativity at night
I'm also a Mozilla tech speaker and I
also organize CSS conversion Tina so by
the way my name is actually all that but
as long as you call me a by Eva or a
face okay well what is this thing this
is something called rotoscopy it's an
animation technique that has more than a
hundred years old and this animation is
no release to make it takes so much time
the thing is that it's really
interesting and even though it has a
hundred years old and it began in the
movies and cinema it has eventually
gotten into the music video industry
especially with this song in 1985 but
also in this another one by Green Day in
2009 Lily Allen 2015 I mean not Paramore
occasionally in two years ago and part
Murat all again in 2018 and why not
let's mention that and Justin Bieber
this year so how does it work well
rotoscoping consists on recording a
completely normal video and drawing on
top of that so you have to tell from 15
to 25 images per second that's a lot
and you can be drawing like for five
hours when you actually play the video
and you have like three seconds of
animation it's not a much but it will be
the best three seconds of your life so
you can turn something like these which
looks really what is going on and -
something like this which it looks
amazing so right now you can do
rotoscoping with other with photoshop
and welcome but we can actually do it in
the browser so the thing is two years
ago this video came out is called hard
times by Paramore and when I took a look
at this it was like this light bulb that
turns on and it's like maybe we could
like thing what we can do right now in
web animations and imperator see what
else we can do the thing is I kind of
took a look at what we have right now in
the web of traditional animation because
once again we keep on bringing things
from tradition and animation from movies
and from the cinema into the web and
it's really interesting how far we have
gotten for example we can do today
just emotions and motion graphics in the
web if you are into learning a
stop-motion you should actually Google
Rachel my words because she's an amazing
teacher and she has so many videos
online and courses as well teaching how
to make a stop motion animation on the
web and when it comes to motion graphics
you can go Chris Connor who is a really
nice person and he does this wonderful
useless animations as well like a pig's
eye spinner so when you're waiting for
the website too long but it looks like a
pizza is great and also I have to
mention Julian must solve our problem
miss rennolds is her name and she
actually did this great thing which is
kind of an and motion graphic animation
but in pure CSS so take a look at her
code but if you can because it's amazing
so my question is can we go further can
we do something else besides this kind
of too traditional image
can't be Mabel does skipping on the web
well the answer will surprise you know
we can and the reason why is because
rotoscoping it's something completely
handmade it has like the soul of the
people who is drawing and I love my
computer is called Minerva of course he
has a name but it doesn't have a soul so
it's not the same you cannot really
pretend to do something that the human
can can do but anyway we can also
interact with the computer and if we
take a look at music videos we cannot
interact with music videos it's like you
watch it and that's it so there is a
place for improvement there is a place
to do something interesting in there we
can take the basic idea of rotoscoping
and take it into the browser so if we
take a look at these part of the video
of heartens by Paramore we can see that
there are free layers we see a
background layer with kind of blue 1
yellow moving things then there is a
second layer in that second layer we'll
see these two people dancing and on top
of them we got a rotoscoping we got
these lines that are moving around and
if you look at the back and like behind
the scenes we got this which is really
funny to watch what they did was
actually they recorded this video they
reverse it so it's actually reversed
they remove that green background fill
it with this kind of yellow and blue
animations put themselves and on top of
that the other animations so we can do
something like this we can begin with
this we can remove the green background
and add something else there is
something called chroma key and it's
really common in movies like Avengers so
what they do is they record a movie with
a green or blue background white green
and blue or not because it's Percy B
well we cannot really use well because
our faces have a tendency to be red so
we have to choose either green or blue
which has the two colors we don't
usually have on our skin and of course
if we will use
the green background we don't wear a
green if we use blue we can wear please
and once you've got that once you got
that blue or green background you need
really good light and I mean really good
light
and that means use not natural light
natural light is like that level for
animations if you have ever work on in
stop-motion animation or even work on
something like chroma key you will
understand that natural light is
horrible simply because these kind of
animations take so much time that maybe
you expect any towers working on it and
in those eight hours the earth moves
around the Sun so suddenly only shadows
move and that is a disaster so we don't
want really this kind of natural light
we prefer these lights whenever we work
with animations or in this case with
chroma key so let me see
chroma key now let's make chroma key
with the users camera because again we
are in the web we can actually enjoy
this so to do this what we need to do is
to take the users input camera and take
that input and put it into a video tab
now we cannot edit a video tag so we
need to copy that into a canvas and then
process that canvas and what we will do
is remove the green or the blue and
replace that with something else so
first step take the users camera input
to do this we actually use getusermedia
which is really simple to use and it's
available a loss in every browser this
will return a promise and then we can
work on it
so we got our B data with an ID of video
and we copy all this streaming from the
camera into that video and if it doesn't
work we just throw an error message and
we get something like this anywhere it's
kind of ok so you got your video tag you
accept the job prompt assess would you
allow using your video and perfect so
let's copy that into a canvas tag so we
get to create our canvas with an ID of
canvas because I'm really creative and
then we basically allow 2d context and
we create a function like this so what
this function will do is it will take
all the information from our video from
the top left corner on to the bottom
right corner and it will copy it into
our canvas now if you only use this code
we will have one issue that is that only
we will see the first frame of the video
copy into the canvas and then the video
will keep on a streaming but it canvas
one refresh so we need to refresh the
canvas all the time there are so many
great performance talk in this
conference that's why mine is not
performant we are going to do a set time
out of zero so we do it the time out the
zero we get something like this we get
the prompt will you share your camera
with this website years and then we get
the videos on the left the original
video and on the right the canvas and
something else we get my cut so it works
it works perfectly and it refreshes all
the time so next is to process the
cameras is to remove the green or the
blue for that we need to do something
called pixel manipulation so we need to
actually kind of remember a little bit
about what is a pixel now I know this is
a front-end conference but let's take a
closer look anyway so a pixel is the
tiniest part of a video or an image like
this a pixel has information in this
case or shibi information so if we take
a look at this pink we are actually
taking a look at this red blast screen
plus this blue and all those free audio
returns this pink
an RGB goes from 0 to 255 0 being black
and 255 being the strongest rather
stronger screen and the strongest blue
so in this case we got a really strong
red a kind of dark green and a kind of
in the middle blue under so through the
alpha Channel and I know this sounds
weird because there is not such a thing
as transparency in videos but anyway
when we ask
information in will return the alpha-2
so what we will do right now is we will
ask for every pixel information of every
frame that our video is getting and we
will ask for the red value of the green
value w value and the Alpha value and
then we will turn that alpha value into
zero so that will be the trick so we
find the green pixels and we'll read
them with them to find the good pixels
we need to take a look at the
information of the video so to do that
we use get image data get image data
will return all the information from
every part of the video in an RGB a way
plus it will return the height and the
width of the video so when I cancel not
that I get this which is okay the width
of the video is 480 the the height is
480 the with a 640 and here it is the
1.2 million array because yeah why
wouldn't you need the 1.2 million all
right when is this 1.2 million because
it seems like a random number it's
actually the width of the video multiply
the height of the video multiply 4
because 4 is the number of information
that you get from each pixel the red the
green the blue and the alpha so if you
open that if it is so you open it again
and again and again and suddenly you get
to the important part so in this part it
looks like random numbers again but we
can find a sequence the sequence is 255
so we know that video doesn't have
transparency so that must be the Alpha
and if that is the alpha that means of
we can identify the others is rgbi your
GPA you should be a of every pixel
beginning from the top left so if we
take a look at the first pixel we get
this value which is equal to this her
GPA so what get there is something like
blueish
yeah because it's 55 red 74 of blue and
86 of green of sorry 74 of green and
ethicists of blue 16 of lewisham is
really dark
those are really low values and the one
of the alpha so let's go through all
that data and let's find the green
pixels and we need to identify them we
identify the red values the green values
and the blue values and then we ask if
the Green is greater than the blue and
it will Green is greater than the red if
that is the case I'm going to turn the
opacity value to syrup and then I have
to return this new information to our
cameras so this gets us something like
this which looks really bad I mean it
works but we cannot really happy on
celebrating because of that so we need
to improve it there are two things that
are really wrong in there first one is a
right that is natural light that's why I
say natural light is not the devil you
don't do natural light you can see that
it's like a lot of light coming from one
side and nothing from the other and then
we need to remove the colors only if the
greens are bigger than 100 why because
as you can see part of my hair has been
taken away I mean hair is not green as
far as I know but it's brown and
basically browns and grays have a value
of red green and blue so even though if
those values are really low the computer
will is still identified it's only has
only one more pixel screen and it will
pick it a screen even though we listen
so whenever it is too dark whenever the
value is too close to zero we really
cannot trust it so we won't remove
values that are lower than 100 because
those are dark values and we are not
sure if they are really green off or if
they art may be grounds or grace so we
get something like this in return so I
kind of worked better we have now no
natural light we have to remove only the
greens if they are bigger than 100 so as
you can see my hair is normal again but
there is something wrong with my t-shirt
I know what you're gonna say it's a
Harry Potter t-shirt it's magic
yes but also is white and white has a
tendency to take in a lot of the colors
from the surroundings so I could either
modify my cold or change my t-shirt
guess what
and I changed my t-shirts that's why I'm
running back today this is what every
developer week at home so let's try it
we'll die let's see so I got these will
you share your coming around
I got our goat cheek with two cameras I
got the camera from the computer and
they lend me this one so I'm going to
use this camera which is much better
Here I am so this is the video on the
left from the canvas on the right and I
made this kind of drop-down where you
can select if you want to filter green
or blue it's in green Rina so we got
this
when we open weave and when you can
please know how works because of the
background thank you and if you turn it
into blue you can see the same but in
all the filters blue are not no longer
green so that's pretty cool but let's
make something even cooler which is
basically invisibility cloak because if
you're like Harry Potter why wouldn't
you make an invisibility cloak so how
does it work well you need to take a
picture of your background without you
so I'm going to move over here let's
take this tell you that and I'm going to
take a picture of that and then I'm
going to use that as a background
instead of the hard words so let's take
this picture the one I just took I'm
going to copy it into our images folder
right next to her words let's go up in a
name and let's modify our beautiful CSS
so one longer it will not no longer be
hard words will be windows some fan set
so we can refresh
and we get that that's lack of any mesh
that's great we are going to allow the
camera see did I use the right camera
that's my question yes there's a right
time around that's good well she's the
other camera over here alright and we
are filtering green so we're twin and
the thing you need to do is like to
cover yourself and that's it it's pretty
simple it's like it's done and invisible
[Applause]
thank you very much
it also works with blue because you
always anytime you get a chance to make
get silly of your selfies like please do
so so it's the best dress you can get
for a conference when you are an
introvert so one final point over here
which is what happens if I want to make
something else what happiness we we
don't only not only want to modify the
backgrounds that's my fault like so more
victim manipulation I can paramour video
in this part the tap if they had the
singer is dancing and at the same time
there are cheaper she would suffer
dancing as well with the same moves so
there is rotoscoping somebody actually
draw that and copy it right next to her
so we can do something like that I'm not
going to dance that would be awkward
they know so if it dancing so we are
going to grab this gift of Taylor Swift
we are going to remove the background
and then we are going to copy Taylor
Swift so now we get free Taylor Swift I
I also kind of shrink that you all are
Taylor Swift so it gets a kind of nicer
effect but you don't really need to do
that
so to copy the Taylor Swift what you do
is create another canvas and then you
position them with position relative and
absolute then after you copy them you
need to make them pink so what you need
to make being is not the green area
because the green is gone you need to
make pink what is left of the green so
we used to have an ephah statement we
are going to use the else
so if what is left is not a green pixel
what we are going to do is we are going
to grab the zero value on the number two
value and the zero value number two are
the information from the green and the
blue so if you put a high value of green
of bread and a high value of blue what
you get is kind of pink so high
Plus hi blue give us pink in the second
and said also found in the third Taylor
Swift so you get something like this
which is Taylor Swift dancing and pink
leadership decides and if you want to
make it better
you cannot kittens because they always
make sense better and since we have any
JavaScript conference it can only act
logos because why wouldn't you so I know
it's not what the scope in because like
yeah and it's a computer it's not a
human doing that but it's pretty cool
and you can do a lot of stuff we can
really open your mind to whatever you
want and he was in your browser I mean
when I was a child and I used to like
this kind of green screen things I used
to actually pirate Adobe Premiere and
Adobe After Effects because I couldn't
afford noise like 15 so I no longer
think that now I got my own software now
I got the browser and I can do that we
shall expect so thank you very much for
listening I will leave you with these
links if I were you I wouldn't open the
second one on my phone because he
crushed mine is actually the online
version of that so I hope you enjoy
thank you very much
[Applause]