Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
I can’t believe that I am saying but, but today
we are going to see how NVIDIA is getting closer and closer to solving an almost
impossible problem. And that is, writing real-time light transport simulations.
So, what is that? And why is it nearly impossible? Well, if we wish to create a truly photorealistic
scene, in computer graphics, we usually reach out to a light transport simulation algorithm,
and then, this happens. Oh yes, concept number one. Noise! This is not photorealistic at all,
not yet anyway. Why is that? Well, during this process, we have to shoot millions
and millions of light rays into the scene to estimate how much light is bouncing around,
and before we have simulated enough rays, the inaccuracies in our estimations show up as noise
in these images. This clears up over time, but it may take from minutes to days for this to happen,
even for a smaller scene. For instance, this one took us 3 full weeks to finish. 3 weeks! Ouch.
Now, earlier, we talked about this technique which could take complex geometry, and 3.4 million light
sources, and it could really render not just an image, but an animation of it interactively.
But how? Well, the magic behind all this is a smarter allocation of these ray samples
that we have to shoot into the scene. For instance, this technique does not forget what
we did just a moment ago when we move the camera a little and advance to the next image. Thus,
lots of information that is otherwise thrown away can now be reused as we advance the animation.
And, note that even then, these smooth, beautiful images are not what we get directly,
if we look under the hood and look at the raw result that comes out of a simulation, we
get something like this. Oh yes. Still, a noisy image. But wait,
don’t despair! We don’t have to live with these noisy images, we have denoising algorithms
tailored for light simulations. This one does some serious legwork with this noisy input.
And, in a followup paper, they also went on to tackle these notoriously difficult photorealistic
smoke plumes, volumetric bunnies, and even explosions interactively. The results were once
again, noise filtered to nearly perfection. Not to perfection, but a step closer to it than before.
Now note that I used the word interactively twice here. I did not say real time.
And that is not by mistake. These techniques are absolutely fantastic,
one of the bigger leaps in light transport research, but, they still
cost a touch more than what production systems can shoulder. They are not quite real time yet.
So, what did they do? Did they stop there? Well,
of course not, they rolled up the sleeves and continued. And now,
I hope you know what’s coming. Oh yes! Have a look at this newer paper they have in this area.
This is their result on the Paris Opera House scene, which is quite detailed, there is a
ton going on here. And, you are all experienced Fellow Scholars now, so when you see them flicking
between the raw, noisy and the denoised results, you now know exactly what is going on. And, hold
on to your papers, because all this takes about 12 milliseconds per frame. Yes yes yes! My goodness!
That is finally in the real time domain and then some! What a time to be alive!
Okay, so where is the catch? Our keen eyes see that this is a static scene. It probably can’t
deal with dynamic movements and rapid changes in lighting, can it? Well, let’s have a look.
Wow! I cannot believe my eyes. Dynamic movement, checkmark. And here, this is as much changing in
the lighting as anyone would ever want, and it can do this too. I absolutely love it.
And, remember the amusement park scene from the previous paper? The one with 23
million triangles for the geometry, and over 3 million light sources?
Well, here are the raw results, and after denoising, this looks super clean. Wow.
So, how long do we have to wait for this? This can’t be real time, right? Well,
all this takes is about 12 milliseconds per frame. Again. And this is where I fell
off the chair when reading this paper. Of course, not even this technique is perfect,
the glossy reflections are a little blurry at places, and artifacts in the lighting can still
appear. But if this is not a quantum leap in light transport research, I don’t know what is.
Plus, if you wish to see some properly detailed comparisons against previous techniques,
make sure to have a look at the paper. And, if you have been holding on to your papers,
now, squeeze that paper, because everything that you see in this
paper was done by two people. Huge congratulations Chris and Alexey!
And, if you are wondering if we ever get to use these techniques,
don’t forget that their marbles demo is already out there for all of us to use.
And it gets better, for instance, not many know that they already have a denoising technique that
is online and ready to use for all of us. This one is a professional grade tool right there. This
is really incredible, they have so many tools out there for us to use, I check what NVIDIA
is up to daily, and I still quite often get surprised about how much they have going on.
So, finally, real time light transport in our lifetimes?
Oh yes, this paper is history in the making.
Thanks for watching and for your generous support, and I'll see you next time!