Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Today we are going to see how crazy good NVIDIA’s new system is at simplifying virtual objects.
These objects are used to create photorealistic footage for feature-length movies, virtual worlds,
and more. But, here comes the problem. Sometimes these geometries are so detailed
they are prohibitively expensive to store and render efficiently. Here are some examples from
one of our papers that were quite challenging to iterate on and render. This took several minutes
to render and always ate all the memory in my computer. So, what can we do if we would still
like to get crisp, high-quality geometry, but cheaper and quicker? I’ll show you in a moment.
This is part of a super complex scene. Get this, it is so complex that it takes nearly a hundred
gigabytes of storage space to render just one image of this and is typically for benchmarking
rendering algorithms. This is the Nurburgring of light transport algorithms if you will.
Well, hold on to your papers because I said that I’ll show you in a moment
what we can do to get all this at a more affordable cost, but,
in fact, you are looking at the results of the new method right now.
Yes, parts of this image are the original geometry,
and other parts have already been simplified.
So, which is which? Do you see the difference? Please stop the video and let me know in the
comments below. I’ll wait. Thank you. So, let’s see together…yes,
this is the original geometry that requires over 5 billion triangles. And, this is the simplified
one, which…what? Can this really be? This uses less than 1 percent of the number of triangles
compared to this. In fact, it’s less than half a percent. That is insanity.
This really means that about every 200 triangles are replaced with just one triangle,
and it still looks mostly the same. That sounds flat out impossible to me. Wow.
So, how does this witchcraft even work? Well, now, you see,
this is the power of differentiable rendering. The problem formulation is as follows. We tell the
algorithm that here are the results that you need to get, find the geometry and material properties
that will result in this. It runs all this by means of optimization, which means that it will
have a really crude initial guess that doesn’t even seem to resemble the target geometry.
But then, over time, it starts refining it, and it gets closer and closer to the reference.
This process is truly a sight to behold. Look how beautifully it is approximating
the target geometry. This looks very close, and is much cheaper to store and render.
I loved this example too.
Previously, this differentiable rendering concept has been used to be able to take a photograph,
and find a photorealistic material model that we can put into our simulation program
that matches it. This work did very well with materials, but, it did not capture the geometry.
This other work did something similar to this new paper, which means
that it jointly found the geometry and material properties. But, as you see
high-frequency details were not as good as with this one. You see here - these details are gone.
And, now, just two years and one paper later, we can get a piece of geometry that is so detailed
that it needs billions of triangles, and it can be simplified two hundred to one.
Now if even that is not enough, admittedly, it is still a little rudimentary, but it even works for
animated characters. I wonder where we will be two more papers down the line from here?
Wow. Scientists at NVIDIA knocked it out of the park with this one.
Huge congratulations to the team! What a time to be alive!
So there you go, this was quite a ride, and I hope you enjoyed it at least half as much as I did. And
if you enjoyed it at least as much as I did, and are thinking that this light transport thing
is pretty cool, and you would like to learn more about it, I held a Master-level course on this
topic at the Technical University of Vienna. Since I was always teaching it to a handful of motivated
students, I thought that the teachings shouldn’t only be available for the privileged few who can
afford a college education, no-no. The teachings should be available for everyone. Free education
for everyone, that’s what I want. So, the course is available free of charge for everyone,
no strings attached, so make sure to click the link in the video description to get started.
We write a full light simulation program from scratch there,
and learn about physics, the world around us, and more.
Thanks for watching and for your generous support, and I'll see you next time!