Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Today we are going to see that the research papers that you see here in these videos are real. So
real that a ton of them are already seeing use in NVIDIA’s AI projects, some of which are so
advanced they seem to be straight out of a science fiction movie, and even more of them are coming.
We will talk about NVIDIA’s GTC event and as always, we will try to have a scholarly angle on
it. For instance, in our Tesla AI Day recap video, we marveled at how transformer neural networks,
a paper from 2017 saw use just a couple years later in real self-driving cars
deployed all around the world. That kind of tech transfer is extraordinary.
So, around here, we talk about a ton of amazing research papers, and under almost every video,
I see some of you asking, okay, this all looks great, but when do we get to use it? That
is completely understandable question, and finally, you’ll get your answers today.
And it gets better, we will also learn three lessons. Lesson one is that AI
is everywhere today and it is improving incredibly fast. For instance, it can now train these virtual
warriors from scratch, or, as many of us have these computers with weird camera placements,
it can recreate our image such that it appears that we are holding eye contact.
Yes, that’s right! Parts of this image are synthetic, parts are made on the fly by an AI,
and it is already almost impossible to notice. And this is FourCastNet. This is a physics model
that can predict outlier weather events, and it runs not in a data center anymore,
it runs on just one NVIDIA graphics card. That is fantastic, lesson two, AI is opening
up completely new frontiers that we never even thought of. And now, hold on to your papers,
and check this out. They can now also create a virtual world where they can accurately simulate
infrastructure projects, down to the accuracy of a millimeter.
When talking about digital copies of real things, this is a virtual assistant, which understands
English, the questions we ask it, and let’s be honest, today, all that is expected. However,
what is not expected is that it can synthesize and answer in a real persons’s voice,
this person is Jensen Huang, the CEO of NVIDIA, and it also animates its mouth and gestures
accordingly. And all this in real time. Look at the face of this proud man! This is priceless.
And, we talked briefly about Tesla, but do you know that NVIDIA is also making progress on
self-driving cars? This is how it sees the world. Marvelous. But, the self driving part is nothing,
watch carefully, because here it comes! Oh yes, lesson number three: everything is connected.
See the assistant there? It sees and identifies the passenger, understands natural language. It
also understands which building is which and what shows are played in them today.
This is self-driving cars combined with the virtual assistant, but, look!
Oh my! This is also combined with virtual copies of real worlds too. By the end of 2024,
they expect to have a virtual, video game version
of all major North American highways, plus Europe, and Asia. What in the world! Wow.
So, I hear you asking, what are all these virtual video game worlds good for?
Why do we need a video game world? Why not just use the real one? It’s right here! Well,
one of the answers is domain randomization. Here, we can reenact real situations for self-driving
cars, and even create arbitrarily complex new situations, and these AIs will be able
to learn from them in a safe environment. You see, once again, everything is connected.
And there is still so much more to talk about, my goodness. For instance, they also have this cell
visualization system that can show us real cells splitting right in front of our eyes
in real time. Not so long ago such a simulation would have taken days, and now,
it runs in real time. They can also create a digital version of real fulfilment centers.
This virtual world can be simulated and optimized before the real one is changed. Does this mean
that? Yes, that is exactly right - now, companies can find the optimal layout for their plans before
making any physical investments. Kind of like the game factorio but in real life. How cool is that?
Their vision system can also look at conveyor belts and adjust their speed based on congestion.
And once again, you see that everything is connected, the self driving engine can also be
put not only into a car, but into a little robot, and thus, it can now navigate these warehouses.
So, whenever you see a new research paper, and it does not immediately show how it could be used,
please do not forget - everything is connected.
They also have their Isaac Gym system,
in which these robots can train safely before they are deployed into the real world.
And, these simulations can get insanely detailed and accurate. For instance, here,
the physics and connections of 5400 parts are simulated, and as a result, this virtual robot
works exactly as the real one. This application would have been unfathomable just a few years ago.
And, yes, this has huge ramifications.
For instance, in the future, whenever we have this robot called Anymal pass a test in a simulation,
it is very likely that it will pass in the real world too.
A simulation that is nearly the same as reality. Just think about that. What a time to be alive!
And once again, applications like this require light transport simulations, cloud streaming,
and collaboration with an AI assistant. And now, you can even stream these from the cloud if you
don't have a beefy graphics card at home. And the results are absolutely amazing. Here you see what
this place will look like at noon. Now, how about some trees nice and maybe more variation. We don’t
even need to be experts in 3D modeling, we just say what we wish to see to the virtual assistant,
and, there we go! We can also ask, what does this look like at night - it looks spectacular.
All that is truly amazing. My eyes were popping out like these machines when seeing
these results for the first time. And, with all of these remarkable results, we really
just scratched the surface. There is so much going on, it is almost impossible to keep
track of all of these amazing projects. Here are some more for your enjoyment.
So, the papers that you see here in this series are real. As real as it gets. We can get from a
research paper to a real product in just a couple years. And everything that you saw here today
is already in production, or will be in production in the near future. By the way
if everything goes well, I will hold my own talk in this year’s GTC as well, and publish
a video of it on this channel. Make sure to subscribe and hit the bell icon to not miss it.
So, what do you think? Does this get your mind going?
What would you use these for? Let me know in the comments below!
Thanks for watching and for your generous support, and I'll see you next time!