Today, I'm going to show you
how to remove anything from 360 immersive
video with the brand new content-aware
fill in Adobe After Effects 2019.
You can remove the camera operator, drone,
360 rover and even
the annoying coffee intern that always
walk into your set in the wrong time.
Before content-aware fill,
the only way to remove objects is using mocha VR.
Now, removal gets a lot easier and even I can do it.
Get a cup of coffee and let me show you how.
[Music]
Hey, what's up everybody?
Your boy Hugh here from CreatorUp.
Right now we have a clip here, one is something that I call drone pole.
What I did is I carry a really long pole and then I'm just
walking but pretend it is a drone, because as you see
this for this footage right here, if I play it, I am at
Washington Monument, the Lincoln Memorial, short clip.
I'm not allowed to fly a drone over there.
Basically it's really long drone pole with a camera right here
and just walking over it, make it look like we are flying.
The whole point is, I need to remove myself, which
is the operator, example one, it's very common.
Second example right here, a very typical
scenario, yes, it's actually a rover.
That is actually from our Meet The Determined
determined project for Special Olympics documentary
and we have the Insta360 pro 2 on a rover
that you see right here, checking over it.
We need to remove the rover.
This is a pretty complicated shot with shadow stitching errors
and reflection, we see how good is content-aware fill on this scenario.
We also have a typical drone shot right here.
If I play this, look up, it's a giant drone right here.
We are flying on top of Athens at Greece.
That's really pretty view.
We need to remove the drone.
This is a three more popular job requirement for any 360 pro
production to remove the drone, the operator and the rover.
Let's take a look at each clip to see how to actually remove all of them.
We'll click back, we'll go ahead
and replace with after effect composition.
We open up an after effect right here.
After open after effect, we'll go ahead and take a look at the footage.
Step one on any 360 footage is to actually see the 2D of this footage.
Go ahead if you haven't already, open VR comp editor.
Go ahead and open that.
Then right here, to go ahead and add a 2D Edit.
That is a very common workflow for any
360 video editing in After Effects.
If you don't know what I'm talking about,
refer back to my old tutorial here.
Go ahead and add 2D Edit, pick the comp right
here and go ahead and hit add 2D Edit.
Now we can pick the camera tool and pen down.
You see me walking right there just
probably trying to find the center point.
You can go ahead, drop the resolution to quarter.
Go ahead and see this.
This is me walking in the Lincoln Memorial.
By the way, I post this video on
Instagram and on the professional group.
People will think that I'm flying a drone
in Lincoln Memorial and get really angry.
Sorry guys, that is a bad prank joke.
I'm not flying a drone in Lincoln
Memorial, I am actually faking a drone
but it looks so good that people think
that I'm actually flying a drone.
That is a technique I'm going to show you right now.
Let's go back to the frame number one,
I'm going to draw a mask around me.
Let's go back to full so I can see what's going on.
Go ahead and click mask.
Click the comm layer, go ahead and draw a garbage mask around me.
I am moving, so go ahead right here, open up mask here,
pick the none so you see the entire frame and the mask.
After you draw the mask, now go ahead
and open mask, make sure you click mask path.
Hit a keyframe right here and then move it.
For the frame one, cover myself.
Let's go ahead and play the footage and just manually move
the mask to make sure the garbage mask is cover me
in each frame. You see, each time I'm inside a keyframe
in the position of the mask path.
This is actually will get really interesting,
because usually in mocha VR, mocha Pro, this is
actually pretty hard track to remove to see if how
good is the content-aware fill compared to mocha.
Remove. That's done.
Let's make sure the last frame, I'm still in frame.
You actually don't need to manually track, this is a really simple shot.
I just throw in a garbage mask to track myself,
but if you want to be more accurate tracking,
I will highly suggest check out an artist secret
weapon of mine, the mocha Pro right here.
Use it and you can have a really accurate
tracking if tracking is your issue.
Let's go ahead and turn the mask back to subtract.
Now you see the mask.
Then if you don't see content-aware fill, go ahead scroll down
a window and it should be right here, content aware fill.
If you don't see that, meaning that you are not
in the latest after effect 2019 spring version.
Make sure you update that.
By the way, I'm going to NAB next week,
and this is actually big announcement on NAB.
Hopefully I will see you guys at NAB.
Go ahead and I like you should have some Alpha expansion
for the mask, you should pick around 15, 16, pretty good.
I also love to add a feather around the mask.
That's a little tip for you guys so we'll have a better smooth remove.
Go ahead and add some feather.
I think that's about there should be good.
We do the same thing in mocha Pro as well.
This is just experience I learned from using Mocha.
Then go ahead and pick object, because
that is an object, I am a object.
If you trying to track remove a surface,
like a logo on your MacBook Pro or any logo
on any machine removing a camera, you use
the surface, it's probably easy to track.
If anything, if these two doesn't work, try edge blending.
Here is entire walking area and then we
just go ahead and generate fill layer.
The good thing is, why after that's doing
that it just like 3D tracking tool.
You can continue working on your next shot and just
in the background let aftereffect do its thing.
Also you see that this is realtime, it's
really fast thanks to the brand new
machine I got from MainGear with the brand
new Intel i9 and also the GTX 2080 TI.
I have this amazing speed.
If you want to check out my machine, I will send a link right here.
See, it's already done, now it's rendering.
As you see, content-aware fill actually generate a JPEG layer
on top of my layer right here as you see and I am already gone.
Look at that, just like magic.
How cool is that?
Now go ahead and do a RAM preview in full
resolution because I have the machine to do it.
Oh my God, look at that, it's perfect remove, you can't see anything.
It's the last frame, you might see a little
bit thing, actually no, I can't see anything.
It's a perfect remove.
For those of you guys, if your clip is
start right in the beginning and the end,
if you see premiere, actually this clip
is not started from the very beginning.
I actually have live before and after frame
to let After Effects before and after.
If you see weird stuff in the beginning or the end of the frame,
just make sure that you pick only the middle part to use.
If your footage doesn't have enough frame so far after you've
had a look before and after, you will not have a gray removal.
This is not magic, it's After Effects
looking for pixel to remove your footage.
That's another tip for you guys.
They look amazing.
The last step is go ahead and back to VR
comp, open output render, boom, I'm gone.
You just go ahead and render this out then you're done.
Now next, look at the pretty complicated rover
shot right here, see if we can handle that.
Go ahead, again, replace with After-Effect composition.
Go ahead, and same idea, go ahead and add a 2D.
Great naming job here.
That's pretty confusing but it's camera number two.
Go ahead and add 2D Edit.
Same idea, go ahead and pick a camera two [?] rover shot.
Now, why this is a more complicated shot,
because well, this shot with a Insta 360 Pro
and there's a stitching involved, you see
stitches here everywhere in this footage.
I should do a better job at stitching but I'm lazy.
So let's see how good is removal done by
the Content Aware Fill from After Effects.
Go ahead, same idea, Pen Tool, go ahead and draw
a garbage mask. Tag over the mask, make sure that is none,
go ahead and just manually track it.
By the way this is a pretty high resolution.
This is 8K by 8K stereoscopic.
I'm just looking at one eye right now.
My machine still play real time at full, no drop
frame, that's a pretty beefy machine I have here.
Again, go check out MainGear, that is their machine.
It's pretty amazing.
The mask is pretty good, because the camera is right underneath the rover.
There's no extra stuff I need to do, just no keyframe.
The whole thing just right there.
Okay, moment of truth.
Now let's go ahead and tag our open Content Aware Fill.
It's done, but the result is pretty horrible.
This one's look like actually Content Aware fill is not doing so well.
Let me go ahead and hide this layer.
Let me try a second time.
This time, I try to get it as close as possible to the rover,
and then I'm also going to try to use create reference frame.
Let me go ahead and see.
Create a reference frame.
I open the frame in Photoshop.
Right here, I'm actually going to paint out the rover right here.
Speaker 1: Okay, go ahead and save that.
Close it.
Now we should have a reference frame right here.
Let's go ahead and create one more reference frame right here.
You see a reference frame,
you actually tell a reference frame number right here,
so you know which frame, your two different frames.
More reference frames, we've going to be a better removal.
That same principle is going to apply on Mocha
as well, that's how you do a perfect removal for
Mocha, is actually generate reference frame, but
Mocha have actually better system to do that.
I think this is good enough.
Go ahead and save it.
Close it.
Okay, now let's try again.
It's actually a great break down to see
how the actual Content Aware Fill work.
If you generate reference frame from frame number
one, to me is frame 37, and then frame 90.
As you see that Content Aware Fill, we use the reference frame to move
between the first frame, to the middle frame, and to the last frame.
If you play that, it's not perfect but
a lot better than what we had before.
If you see what we had before, that's what we had before,
and that's what we have after, just by inserting
three keyframe as a reference frame.
For this really challenging shot, the reason why it's
we got horrible stitching job, different lighting,
the flashing surface and complicated background, and also
so close to the object, and the remove area is so big.
I would highly suggest to just go ahead and generate more
reference frame in Photoshop and then use that to move
between frame and you will get a perfect result if you
do the work, but at least now you have a way to do it.
Again, if you have Mocha Pro and Mocha removal
[?], this process will be way much easier.
I will show you the same result from Mocha Pro at
the end, but let's move on to the last shot, which
is amazing drone shot and we need to remove a pretty
big area, which is the really close up drone.
Go ahead, and again replace with After Effect Composition.
Same idea, go ahead and add a 2D Comp.
This is comp number three.
Go ahead and 2D Edit, select camera two.
That is actually very big.
For this shot is actually very challenge, because
the drone is so close to the camera, even
I use an After Effect VR Comp Editor, I can't
get the full drone in the frame to remove.
This time I'm actually not going to use the VR comp,
I'm just go ahead and remove this entire thing.
To enter layer right down, right there, go ahead and hit Subtract.
Okay, now we are done, but look at the result, it's very impressive.
Before, I actually have it none, that is a drone that
covered the entire sky, and after, it's completely gone.
You come and improve a little bit the result right
here at the cloud, by generate create Reference Frame,
but I just want to show you how powerful is this
brand new Content Aware Fill for After Effects.
Such a break area, you can literally remove and replace the sky.
Again, if you want to have a perfect removal, I would still
suggest you try out Mocha Pro and follow my other tutorial
to get a surgical removal, but this literally will save you
hours of removal time with the brand new Content Aware Fill.
Now I just want to quickly show you why
I say mocha is still very necessary
on removal, especially for very complicated
shot like this one you just saw.
Right now is as you see, the rover is gone, before, after.
It's gone nicely.
Let's take a look at Mocha.
What happened here is I'm using Mocha to track.
We actually highlight all this little area to track the scene
with the BG track, which is lock, and then I have a garbage mask
right here, remove mask, that should BG remove to link to the BG
track layer and then I have to remove mask on top of the rover.
I need to remove.
If you can't see it, you can turn on 360.
One cool thing about Mocha also you can zoom all
the way out to see what is going on for the entire frame.
I go over to the first frame right here, go to remove mask right
here, hold on remove, and go ahead just render the first frame.
As you see, we already have a perfect
remove compared to what we had before.
In Mocha, as I mentioned earlier, is actually
also very easy to create clean plate.
As you see there is a clean plate, I
just click a button and generate for me.
No need to touch Photoshop at all.
If you go ahead and open edit, I only generate four
but not 10 clean plate in zero, 30, 60, 120 frame.
We take a look at all this frame right here.
See again look perfect.
You actually see the line right here
because it's not generated from Photoshop.
I did not do it, it just generate from Mocha and I save that
as an outer frame and then go ahead and hit 60, same idea.
Go ahead and generate a clean plate.
Again, look absolutely perfect.
Look at the straight line, everything looks amazing.
Then we go ahead and just save
the project, we are not going to remove in
Mocha, we are going to remove everything
in After Effect, as you see why now.
The only drawback is Mocha is very slow
and GPU intensive, but as you see,
the removal is perfect, and here is
the before and after from the Mocha removal.
Thank you for watching.
As you all know, I was gone for three months filming for
a couple of pretty big international clients, but now I'm back
and I'm doubling down to make more advanced 360 post production
tutorial for you guys so you can up your game in 360 VR.
To support me making more tutorial for you guys, don't
forget to share this video to your friends and to your boss.
Subscribe if you have not, smash that
like button and support your Boy Hugh so
he don't need to go away again to take on
more DB job just to keep the light on.
I will see you in the next amazing After
Effects or Adobe Premiere tutorial.
[music]