I'm no game designer, but I made a VR character that kinda looks like my wife,
Chary. It speaks like her naturally, and I did this in less than an hour with the help of... AI.
CAS: Oh hello, can you tell me who you are? CHARY:
How could you forget that I'm your wife! CAS: Oh, I'm so sorry for forgetting that
for a second. I love you very much. CHARY: I love you too.
This is pretty game-changing tech.
How did I do this? And what does this mean for VR games in the near future?
That's what I want to explore today in a more experimental video, which I hope you'll enjoy,
and hopefully I can teach you a thing or two about something I personally think is fascinating.
Thanks to NVIDIA for sponsoring this video! Because of this partnership,
I can host a giveaway for an RTX 4080 graphics card, and it's totally free
to participate. Instructions are at the end of this video. Now, join me beyond reality.
Artificial Intelligence is the talk of this year.
We've got ChatGPT, Bing AI, Bard, Midjourney, DALL-E, Synthesia...
These are all AI models able to do things instantly you usually need
a lot of time for. Like spitting out whatever you have in mind for
a complex painting. You can ask AI to help you with coding or write a video
script. It can even generate whole music videos just like that *snaps fingers*.
As I researched for this video, my mind was blown away by what you can do right
now. And it got me thinking about how AI can change VR games or gaming in general.
There is actually one company leading the way on this,
and you can probably guess it by now. NVIDIA. You probably know them from their gaming GPUs,
but they also saw AI coming in this form and have been investing in this
for over a decade already. No one believed it was a sustainable investment back then,
but now every chip they make now is focused on AI, and they are the ones powering ChatGPT.
Which reached 100 million active users two months after launch.
But they are not only doing hardware. Their latest project is Omniverse ACE,
short for Avatar Cloud Engine. It’s a set of developer tools to revolutionize 3D
game characters and avatars. It's a huge step forward for the tech.
Basically, this brings text-based AI like ChatGPT to life with animated faces that
act a lot like humans. NVIDIA believes this is how we will communicate with AI in the future,
and if that works out, this might be the tech we'll use to customize our own appearances in VR.
And get this, it's not something far away in the future. It's here NOW for
free. I used ACE to make my virtual waifu speak like a human, and anyone can do it.
Now, I know AI with a face might sound a bit... Terminator,
but don't worry, NVIDIA isn't Skynet, they're building these tools to make it
easier for devs or anyone really to work on metaverse projects.
And it's not just about AI faces; they're doing something much bigger.
Before telling you how I made the avatar, let me give you some context.
The Omniverse is a real-time 3D design collaboration and digital
twin platform. That's a mouth full, but it's basically what you expect a metaverse to be,
not a walled garden like Horizon Worlds, but something more like Ready Player One.
Yes, the Omniverse is more like one virtual universe that connects to other virtual
worlds owned by others, and the best thing is that it's open, so everything works together.
It’s built to communicate with different software,
made possible by using USD, an open standard initially invented by Pixar. That's right,
Disney's animation studio. This allows you to connect different tools, assets,
and projects so people can collaborate remotely without limiting themselves to one platform.
So USD will hopefully let everyone create complex 3d worlds for web 3.0 and then be able to connect
them with ACE-enabled AI. That's what NVIDIA is after, and I think that's pretty cool.
An example of this is my dear waifu here.
ACE is part of the Omniverse and it has an AI service called Audio2Face. This
generates facial motion and lip sync in real-time.
It even matches expressions and emotions to the words...
And I have complete control over it; I can make it look amazed, angry, happy, or sad.
And then, to make her beautiful, I connected it to Unreal Engine's Metahuman.
But since its the Omniverse, I could have used Unity or Ready Player Me,
the popular avatar creator for VRChat. It doesn't matter where you design;
you can take your avatar from one tool to another and work on it.
And I'm NOT a game designer, so if I can do this, let alone what a game designer can do.
With just ACE they can…
Think about adding Conversational AI, like Riva Speech, so the character can
carry a natural conversation or answer complex questions.
I could even use my voice to say stuff but make it sound like Chary because
it can create a natural custom voice with only 30 minutes of voice data. This means
you can also create a whole new voice to narrate a storyline. While you're at it,
why not generate storylines based on existing game content. Omniverse has an app for that.
Maybe you need some movement? Then you give the AI a random video with someone moving,
and it can estimate that physics-based motion and put into your character.
Or do you need variations of your avatar?
AI can generate them for you by taking an example 2D image.
Now take it further and use AI to translate your voice into multiple
languages in real-time, so you can even use it while video calling.
The possibilities are insane with this. What does this mean for VR games?
One of the biggest things AI can do is help make a
game more engaging and challenging based on the player's personality and skills.
For example, AI can create enemies that are more challenging to defeat,
as they can adapt and learn from the player's actions.
This tech can be used to create NPCs that look and react like real people. It could
perceive the game environment and carry out actions based on your requests. Think about
asking it to carry out a battle strategy or having it show you the way to some area.
AI can be used to generate dialogue or dynamically adjust the storyline based
on a player's choices, leading to more surprising storytelling experiences.
And I'm only talking about avatars here, which only scratches the surface.
There are so many more AI applications to explore, not just NVIDIA’s. Lately,
it seems like they're popping up like mushrooms.
Of course, AI has potential dangers, like with this tech people could
easily impersonate others and it's important to be aware of the risks.
But NVIDIA is on a path to make it easier and more accessible in an open way,
a way I can stand behind. So I hope more developers will start using this tech.
Now I couldn't add everything I wanted like true conversational
AI to my virtual avatars as I didn't have enough time. But I intend to. Let
me know if you'd like to see a video of that and what else you'd like me to add!
If you are interested in checking out Omniverse ACE, I'll put a link below. It's free to use
but currently in early access, so you need to sign up for an account and get approved until
it releases for the public, but I think they are only asking for SOME technical
ability. It should be an easy entry. The only thing REQUIRED is an RTX GPU to run Omniverse.
But if you don't have one, together with NVIDIA, we’re giving away an RTX
4080 graphics card exclusive to our viewers who use our link to register.
All you need to do is:
Subscribe and comment down below what you think about NVIDIA's AI
Register for GTC using the link below. Use your new account to watch at least one
virtual session during GTC on March 20 to 23. You can check it out from the comfort of your home,
as it's all online :) but keep in mind that the keynote is excluded.
GTC is NVIDIA's FREE developer conference is about breakthroughs in AI, the metaverse, and beyond.
There are many things to do during the virtual events, like online training, insights, and access
to experts on the future of generative AI and GPU programming for developers. There are also some
great talks about Omniverse ACE there. I'll link a few interesting ones down below.
You do need a permanent address in Europe, the Middle East, and Africa to qualify for
the giveaway, but I hope you'll register anyway, even if you don't want to win anything, as that
supports us in the way that, hopefully, NVIDIA will work with us more in the future. Maybe with
more giveaways. Cus I'm actually super proud to partner with them. I have used their GPUs
all my life, and they have always brought me so much joy because I could play games.
Anyways, GOOD LUCK, and if you like my talks about future technology,
I have another one right here about the future of AR glasses. Check it out!
Special thanks to my Champs, but love y’all. Like this video if you enjoyed it and VR on!