Dear Fellow Scholars, this is Two Minute Papers with Dr. Károly Zsolnai-Fehér.
Today, through the power of AI research, we are going to see how easily we can ask virtual
characters to not only say what we say, but we can even become an art director and ask
them to add some emotion
to it.
So, how does this happen?
Well, in goes what we say, for instance, me uttering Dear Fellow Scholars, or anything
else.
And, here is the key, we can also specify the emotional state of the character.
And this AI does the rest.
That is absolutely amazing.
But, it gets even more amazing.
Now, hold on to your papers, and, look.
Yes, that’s right, this was possible back in 2017, approximately 400 Two Minute Papers
episodes ago.
And whenever I showcase results like this, I always get the question from you Fellow
Scholars, asking that “yes, this all looks great, but when do I get to use this?”.
And the answer is, right now.
Why?
Because NVIDIA has released Audio2Face, a collection of AI techniques that we can use
to perform this quickly and easily.
Look, we can record our voice live, and have a virtual character say what we are saying.
But, it does not stop there, it also has 3 amazing features.
One, we can even perform a face swap, not only between humanoid characters, but, my
goodness.
Even from a humanoid to, for instance, a Rhino!
Now that’s something.
I love it.
But wait, there is more.
There is this.
And this too.
Two, we can still specify emotions, like anger, sadness and excitement, and the virtual character
will perform that for us.
We only need to provide our voice, no more acting skills are required.
In my opinion, this will be a godsend in any kind of digital media, computer games, or
even when meeting our friends in a virtual space.
Three, the usability of this technique is out of this world.
For instance, it does not eat a great deal of resources, so we can run multiple instances
of it at the same time.
This is a wonderful usability feature, one of many that really makes or breaks when it
comes to a new technique being used in the industry or not.
An aspect not to be underestimated.
And here is another usability feature: it works well with Unreal Engine's MetaHuman,
this is a piece of software that can create virtual humans.
And with that, we can not only create these virtual humans, but become the voice actors
for them without having to hire a bunch of animators.
How cool is that.
Now I believe this is an earlier version of MetaHuman, here is the newer one.
Wow, way better.
Just imagine how cool it would be to voice these characters automatically.
Now, the important lesson is that this was possible in a paper in 2017, and now, in a
few years it has vastly improved, so much so that it is now out there deployed in a
real product that we can use right now.
That is a powerful democratizing force for computer animation.
So, yes, the papers that you see here are real.
As real as it gets, and this tech transfer can often occur in just a few years time.
In some other cases, even quicker.
What a time to be alive!
So, what would you use this for?
I’d love to know what you think.
Let me know in the comments below!
Thanks for watching and for your generous support, and I'll see you next time!