Hi, my name is Nao Adachi. In this video
I will show you how I remove the shadow over a flying drone by using projection mapping and time offset
This technique is useful when the shot is very long
and
if you try to remove something by making patches and tracking you would have to frame hold so many different frames as
You can see if you use this frame to cover up this area
By the time it gets to this point
The pixel information might not be good enough so you would have to keep making
patches by frame holding again and again
other methods like
Content-aware feel from Photoshop and After Effects can sometimes get good result in some shots
But I think this technique can be more accurate and better quality if the tracking is good
First eye 3d tracked the ground and set up a scene with the camera
Here's what I have for the 3d truck
So I set up the scene
I need to align a card with this ground. So I need to make this ground flat
So now I make a card
And align with this ground
Here is what I have
If the ground is not flat
It might work better to create matching objects or a mesh from point cloud generator
For example, this area is flat
Because of this road
But this area might not be flat because of the desert area and the mountain
In this case, I used two cards for the road and the desert area
Next I set up projection mapping onto the card
And the trick is to use time offset
Only for projection map texture and the camera for the projection map. So these two
In these time offsets should be exactly the same so you can just instance it
Also let's set up the scanline renderer make sure you don't use the time offset for the scanline renderer
Next you wrote. Oh just the area you want to remove. In this case, I roto the shadow of the drone
Then I use this roto to mask our projection mapping element and merge it with the original footage
Finally you tweak the time offset so that the shadow will disappear from our projection mapping element
What's happening here is I'm using the previous frame of the camera position
for the projection mapping
but the actual camera
For the scanline render is a current frame
So the texture is projected into the right place and rendered by the current camera
but the shadow of the drone is not there yet
You should add the blur to the roto to smooth out the edge
As you can see the car is moving with the drone so it caused issues
Because the car doesn't require so many different patches like the ground
I just tracked the car and use frame hold for a couple of frames to make a few patches
My truck is not perfect and I just used cars instead of mesh from point cloud generator
So it is a little bit off, but it got the job done
Also drone footage might have lens distortion. So on distorting the footage might work better for tracking
So, please let me know what you guys think or if you have any questions. Thank you for watching