hello and welcome to red stapler channel in this video we are going to show
you the basic of real-time video processing in JavaScript learn how to
remove the green screen and use another video as background ready let's check it out
first I'm going to download a video with a green screen from pexels
then put the video on our page using video attack
I will set a tribute to autoplay mute and loop
as for the output I'll use canvas with the same size as video
here's the current page with original video on the left and blank canvas on
the right side now let's work on the code first I will create an initialize
function
let's start with getting the video and canvas element
when working with canvas we need to work with its context not the canvas element
itself
I will also create another canvas as a temporary working space
we will use this canvas to extract each video frame and process it before we put
it into the output canvas
at the end of initialization we will add eventlistener to start processing when
the video start playing
we will create another function calls compute frame
I'm going to use a temp canvas to get the current video frame as image using
DrawImage method and pass our video element and video size
now the video frame has been painted on a temp
canvas next we will get the image data using get image data method
we will work on the processing later for now just past the image data to the
output canvas
then use set timeout to recursively call itself and create a
rendering loop
finally we will call init function when the page content is loaded
here is our current page the original video is playing on the left and the
output canvas is on the right next we are going to start removing the green
screen background first let's talk about the data structure of the image data the
image data we currently have on the canvas is in a single dimensional array
format it starts with the first pixel off the first row followed by the next
pixels in the same row then start over with the next row repeatedly until we
cover the entire image now each pixel has four datas the first
three is RGB value and the last one is alpha or transparency so each pixel need
four array spaces and final array size will be four times of the actual number
of pixels
to remove the green-screen we need to create loop to check all the pixels RGB value
notice that I have divided over a size by four
then we can get the RGB value for each pixel by multiplying the index by four
and add offset R is the first value for each pixel
hence zero offset G and B will need 1 and 2 offset respectively
I'm going to use a color picker plug-in to check RGB value after green screen
then add if statement to check each pixel if it's RGB value are close to the green
screen color
you can adjust the criteria as you like to broad and you will also remove non
background pixels to narrow and you'll still have green background so you have to
find a sweet spot
then set the alpha value to zero
since this is a basic RGB check you will see that there're still green pixels left
we will need more advanced algorithm to check the color but for this tutorial
that should be enough next I'm going to add another video as
background so I'm going to download this fire effect video from pexels first
I'm going to dynamically create a video element for the background video
and set the source and other attributes
then used a temp canvas to extract each frame like we did with the first video
then instead of removing the green-screen from the first
video we will replace the RGB value with a second video
and that's it let's see result
and that's all for this video hope you guys enjoy if you want to see more dev
tips and tutorials subscribe our channel to stay tuned thanks watching see you
next upload bye