Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Today we are going to do this, and this, and this.
One of these applications is called a NERF.
What is that? NERFs mean that we have a collection of photos like these,
and magically, create a video where we can fly through these photos. Yes, typically, scientists
now use some sort of learning-based AI method to fill in all this information between these photos.
Something that sounded like science fiction just a couple years ago, and now, here we are.
Now, these are mostly learning-based methods, therefore, these techniques need
some training time. Wanna see how their results evolve over time? I surely do, so,
let’s have a look together! This NERF paper was published about a year and a half or two years
ago. We typically have to wait for at least a few hours for something to happen. Then came
the Plenoxels paper with something that looks like black magic. Yes, that’s right, this trains
in a matter of minutes. And it was published two months ago. Such improvement, in just two years.
But, here is NVIDIA’s new paper from about a month ago. And I hear you asking,
Károly, are you telling me that a two month old paper of this caliber is going to be
outperformed by a one month old paper? Yes, that is exactly what I am saying.
Now, hold on to your papers, and look here, with the new method, the training takes…what?
Less time than I have to utter this sentence, because it is already done. So
first, we wait from hours to days, then, 2 years later, it trains in minutes,
and a month later, just a month later, it trains in a couple seconds. Basically, nearly instantly.
And, if we let it run for a bit longer, but still less than two minutes, it will not only outperform
a naive technique, but will provide better quality results than a previous method,
while training for about ten times quicker. That is absolutely incredible. I would say
that this is swift progress in machine learning research,
but that word will not cut it here. This is truly something else.
But, if that wasn’t enough, NERFing is not the only thing this one can do.
It can also approximate a gigapixel image. What is that? That is an image with tons of data in it,
and the AI is asked to create a cheaper neural representation of this image. And,
we can just keep zooming in, and zooming in, and we still find new details there.
Now if you have been holding on to your papers so far, now, squeeze that paper, because what
you see here is not the result, but the whole training process itself. Really? Yes, really.
Did you see it? Well, did you blink? Because if you did, you almost certainly missed it.
This was also trained from scratch, right in front of our eyes. But it’s so quick, that if you take
just a moment to hold on to your papers a bit more tightly, and you already missed it. Once again,
a couple papers before, this took several hours at the very least. That is outstanding.
And if we were done here, I would be very happy. But, we are not done yet, not even close! It can
still do two more amazing things. One, this is a neural signed distance field it has produced. That
is a mapping from 3D coordinates in a virtual world to distance to a surface. Essentially,
it learns the geometry of the object better because it knows what parts are inside, and
outside. And it is blazing fast, surprisingly, even for objects with detailed geometry.
And, my favorite, it can also do neural radiance caching. What is that? At the risk of simplifying
the problem, essentially, it is learning to perform a light transport simulation. It took
me several years of research to be able to produce such a light simulation, so let’s see how long it
takes for the AI to do. Well…let’s see…holy mother of papers! NVIDIA, what are you doing! I give up.
As you see, the pace of progress in AI and computer graphics research
is absolutely incredible, and even better, it is accelerating over time.
Things that were wishful thinking 10 years ago become not only possible,
but are now easy over the span of just a couple of papers. I am stunned. What a time to be alive!
Thanks for watching and for your generous support, and I'll see you next time!