Deep learning has unquestionably revolutionized computing.
Researchers continue to innovate at lightspeed with new models and new variants.
We're driving hard to invent new hardware and systems for researchers to make groundbreaking discoveries.
We're creating new systems to expand AI into new segments - like Arm, enterprise, or 5G edge.
We're also using these systems to do basic research in AI, and building new AI products and services.
Let me show you some of our work in AI, basic and applied research.
DLSS: Deep Learning Super Sampling.
StyleGAN: AI high-resolution image generator.
GANcraft: a neural rendering engine, turning Minecraft into realistic 3D.
GANverse3D: turns photographs into animatable 3D models.
Face Vid2Vid: a talking-head rendering engine that can reduce streaming bandwidth by 10x, while re-posing the head and eyes.
Sim2Real: a quadruped trained in Omniverse, and the AI can run in a real robot - digital twins.
SimNet: a physics-informed neural network solver that can simulate large-scale multi-physics.
BioMegatron: the largest biomedical language model ever trained.
3DGT: Omniverse synthetic data generation.
And OrbNet: a machine learning quantum solver for quantum chemistry.
This is just a small sampling of the AI work we're doing at NVIDIA.
We're building AIs to use in our products and platforms.
We're also packaging up the models to be easily integrated into your applications.
We've created several classes of applications
that you can use as a reference -
for example, Metropolis video analytics,
Isaac robotics,
Jarvis conversational AI.
These are essentially no-coding open-source applications that you can modify.
Now, we're offering NGC pre-trained models that you can plug into these applications or ones you develop.
These pre-trained models are production-quality, trained by experts, and will continue to benefit from refinement.
There are new credentials to tell you about the models' development, testing, and use.
And each comes with a reference application sample code.
NVIDIA pre-trained models are state of the art and meticulously trained,
but there is infinite diversity of application domains, environments, and specializations.
No one has all the data - sometimes it's rare, sometimes they're trade-secrets.
No model can be trained for every skill.
And the specialized ones are the most valuable.
So we created technology for you to fine-tune and adapt
NVIDIA pre-trained models for your applications.
NVIDIA TAO is a framework that encapsulates the entire workflow to customize AI models.
TAO applies transfer learning on your data to fine-tune our models with your data.
TAO has an excellent federated learning system
to let multiple parties collectively train a shared model while protecting data privacy.
NVIDIA federated learning is a big deal -
researchers at different hospitals can collaborate on one AI model while keeping their data separate to protect patient privacy.
TAO uses NVIDIA's great TensorRT to optimize the model for the target GPU system.
With NVIDIA pre-trained models and TAO, something previously impossible for many can now be done in hours.
Fleet Command is a cloud-native platform
for securely operating and orchestrating AI across a distributed fleet of computers -
it was purpose-built for operating AI at the edge.
Fleet Command, running on certified EGX systems,
will feature secure boot, attestation, uplink, and downlink to a confidential enclave.
From any cloud or on-prem, you can monitor the health of your fleet.
Unfamiliar circumstances can be uploaded to form new training datasets.
The fleet can be selectively updated.
For robotics and AV, Fleet Command will allow teleoperations.
Let's take a look at how one customer is using NGC pre-trained models and TAO to fine-tune models
and run in our Metropolis smart city application
orchestrated by Fleet Command.
No two industrial environments are the same. And conditions routinely change.
Adapting, managing, and operationalizing AI-enabled applications at the edge for unique, specific sites can be incredibly challenging -
requiring a lot of data and time for model training.
Here, we're building applications for a factory with multiple problems to solve.
Understanding how a factory floor space is used over time.
Ensuring worker safety, with constantly evolving machinery and risk factors.
Inspecting products on the factory line, where operations change routinely.
We start with a Metropolis application running three pretrained models from NGC. We are up and running in minutes.
But since the environment is visually quite different from the data used to train this model,
the video analytics accuracy at this specific site isn’t great.
People are not being accurately recognized and tracked, and defects are being missed.
Now let's use NVIDIA TAO to solve for this.
With this simple UI, we retrain and adapt our pre-trained models with labeled data from the specific environment we're deploying in.
We select our datasets.
Each is a few 100 images, as opposed to millions of labeled images required if we were to train this from scratch.
With NVIDIA TAO, we go from 65% to over 90% accuracy.
And through pruning and quantization, compute complexity is reduced by 2x -
with no reduction in accuracy and real-time performance is maintained.
In this example, the result is three models specifically trained for our factory - all in just minutes.
With one-click, we update and deploy these optimized models onto NVIDIA-Certified servers with Fleet Command -
seamlessly and securely from the cloud.
From secure boot to our confidential AI enclave in GPUs, application data and critical intellectual property remain safe.
AI accuracy, system performance and health can be monitored remotely.
This establishes a feedback loop for continuous application enhancements.
The factory now has an end-to-end framework to adapt to changing conditions - often and easily.
We make it easy to adapt and optimize NGC pre-trained models with NVIDIA TAO
and deploy and orchestrate applications with Fleet Command.
We have all kinds of computer vision, speech, language and robotics models,
and more coming all the time.
Some are the works of our genetics and medical imaging team.
For example, this model that predicts supplemental oxygen needs from X-ray and electronic health records.
This is a collaboration of 20 hospitals across eight countries and five continents for COVID-19.
Federated learning and NVIDIA TAO was essential to make this possible.
We all talk to AI in the cloud.
Some conversations need to be interactive,
trained for special lingo, protected for privacy, or processed locally to save bandwidth.
The world needs a state-of-the-art conversational AI that can be customized and processed anywhere.
Today, we're announcing the availability of NVIDIA Jarvis -
a state-of-the-art deep learning AI for speech recognition, language understanding, translations and speech.
End-to-end GPU-accelerated, Jarvis interacts in about 100 milliseconds -
listening, understanding and responding faster than the blink of a human eye.
We trained Jarvis for several million GPU-hours, on over 1 billion pages of text, and 60,000 hours of speech
in different languages, accents, environments and lingos.
Out of the box, Jarvis achieves a world-class 90% recognition accuracy.
You can get even better for your application by refining with your own data using NVIDIA TAO.
Jarvis supports five languages today - English, Japanese, Spanish, German, French, and Russian.
Jarvis does excellent translation.
In the Bilingual Evaluation Understudy benchmark, Jarvis scores forty points
for English-to-Japanese,
and fifty points for English-to-Spanish.
Forty is high quality and state of the art. Fifty is considered fluent translation.
Jarvis can be customized for domain jargon.
We've trained Jarvis for technical and healthcare scenarios. It's easy with TAO.
And Jarvis now speaks with expression and emotion that you can control - no more mechanical talk.
And lastly, this is a big one - Jarvis can be deployed in the cloud,
on EGX in your data center,
at the edge in a shop, warehouse, or factory running on EGX Aerial,
or inside a delivery robot running on a Jetson computer.
Jarvis early access program began last May
and our conversational AI software has been downloaded
45,000 times.
Among early users is T-Mobile, the U.S. telecom giant.
They're using Jarvis to offer exceptional customer service
that demands high quality and low latency needed in real-time speech recognition.
We're announcing a partnership with Mozilla Common Voice,
one of the world's largest multi-language voice datasets, and it's openly available to all.
NVIDIA will use our DGXs to process and train Jarvis
with the dataset of 150,000 speakers in 65 languages,
and offer Jarvis back to the community for free.
So go to Mozilla Common Voice and make some recordings -
let's make universal translation possible
and help people around the world understand each other.
Now let me show you Jarvis.
The first part of Jarvis is speech recognition.
Jarvis is over 90% accurate out of the box - that's world-class, and you can still use TAO to make it even better for your application,
like customizing for healthcare jargon.
Chest X-ray shows left retrocardiac opacity, and this may be due to atelectasis, aspiration, or early pneumonia.
I have no idea what I said, but Jarvis recognized it perfectly.
Jarvis translation now supports five languages.
Let's do Japanese.
"Excuse me,
I'm looking for the famous Jangara Ramen shop. It should be nearby, but I don't see it on my map.
Can you show me the way? I'm very hungry."
That's great. Excellent accuracy. I think. Instantaneous response.
You can do German, French, and Russian - with more languages on the way.
Jarvis also speaks with feelings.
Let's try this -
"The more you buy, the more you save!"
“The more you buy, the more you save.”
I think we are going to need more enthusiasm.
"The more you buy, the more you save!"
NVIDIA Jarvis - state-of-the-art deep learning conversational AI,
interactive response, five languages, customize with TAO, and deploy from cloud, to edge, to autonomous systems.
Recommender systems are the most important machine learning pipeline in the world today.
It is the engine for search, ads, online shopping, music, books, movies, user-generated content, news.
Recommender systems predict your needs and preferences from past interactions with you, your explicit preferences,
and learned preferences using methods called collaborative and content filtering.
Trillions of items to be recommended to billions of people -
the problem space is quite large.
There are hundreds of recommendation engines in internet service companies processing exponentially growing datasets.
We would like to productize a state-of-the-art recommender system
so that all companies can benefit from the transformative capabilities of this AI.
We built an open-source recommender system framework called Merlin,
which simplifies an end-to-end workflow from ETL, to training, to validation, to inference.
It is architected to scale as your dataset grows.
And if you already have a ton of data, it is the fastest recommender system ever built.
In our benchmarks, we achieved speed-ups of 10-50x for ETL,
2-10x for training, and 3-100 times for inference depending on the exact setup.
Merlin is now available on NGC.
By the way, Jarvis used for call-center customer support is a fabulous way to generate data that will be used by Merlin
to learn to recommend products and solutions to your customers.
Our vision for Maxine is to eventually be your avatar in virtual worlds created in Omniverse -
to enable virtual presence.
The technologies of virtual presence can benefit video conferencing today.
Alex is going to tell you all about it.
Hi everyone, I'm Alex and I'm the product manager for NVIDIA Maxine.
These days, we’re spending so much time doing video conferencing.
Don't you want a much better video communication experience?
Today, I am so thrilled to share with you the NVIDIA Maxine,
a suite of AI technologies that can improve the video conferencing experience for everyone.
When combined with NVIDIA Jarvis, Maxine offers the most accurate speech to text.
See? It's now transcribing everything I'm saying, all in real time.
And in addition, with the help of Jarvis, Maxine can also translate what I am saying into multiple languages.
That would make international meetings so much easier.
Another great feature I'd love to share with you is Maxine's Eye Contact feature.
All the times when I am presenting, I am not looking at a camera.
When I turn on Maxine's eye contact feature, it corrects the position of my eyes so that I'm looking back into the camera again.
Now, I can make eye contact with everyone else.
The meeting experience just gets so much more engaging!
Last but definitely not least, Maxine can even improve the video quality when bandwidth isn't sufficient.
Now let's take a look at my video quality when bandwidth drops to as low as 50 kbps.
Definitely not great.
Maxine's AI face codec can improve the quality of my video even when bandwidth is as low as 50 kbps.
The most powerful part of Maxine is that all of these amazing AI features can run simultaneously together and all in real time.
Or you can just pick and choose a few. It's just that flexible!
Now before I go let me turn off all the Maxine features so you can see what I looked like without Maxine's help.
Not ideal.
Now let's turn back on all the Maxine features.
See? That’s so much better thanks to NVIDIA Maxine!
NGC has pre-trained models.
TAO lets you fine-tune and adapt models to your applications.
Fleet Command deploys and orchestrates your models.
The final piece is the inference server -
to infer insight from the continuous streams of data coming into your EGX servers or your cloud instance.
NVIDIA Triton is our inference server.
Triton is a model scheduling and dispatch engine that can handle just about anything you throw at it:
Any AI model that runs on cuDNN, so basically every AI model.
From any framework - TensorFlow, PyTorch, ONNX, OpenVINO, TensorRT, or custom C++/Python backends.
With controllable service requirements, whether real-time interactive or high-throughput batch,
Triton schedules on the multiple generations of NVIDIA GPUs and x86 CPUs.
Triton maximizes the CPU and GPU utilization.
Triton scales with Kubernetes and handles live updates.
Triton is for anyone doing scale-out inference on NVIDIA GPUs or x86 CPUs.
Triton is fantastic for everything from image to speech recognition, from recommenders to language understanding.
But let me show you something really hard - generating biomolecules for drug discovery.
In drug discovery, it all comes down to finding the right molecule, the right shape, the right interactions with a protein -
the right pharmaco-kinetic properties.
Working with scientists from AstraZeneca, NVIDIA researchers trained a language model to understand SMILES -
the language of chemical structures.
We developed the model with Megatron and trained it on a SuperPOD.
Then created a generative model
that reads the structure of successful chemical drug-compounds
to then generate potentially effective novel compounds.
Now we can use AI to generate candidate compounds
that can then be further refined with physics-based simulations,
like docking or Schroödinger's FEP+.
Generating with a CPU is slow - it takes almost 8 seconds to generate one molecule.
On a single A100 with Triton, it takes about 0.3 seconds -
32x faster!
Using Triton Inference Server, we can scale this up to a SuperPOD and generate thousands of molecules per second.
AI and simulation is going to revolutionize drug discovery.
We provide these AI capabilities to our ecosystem of millions of developers around the world.
Thousands of companies are building their most important services on NVIDIA AI.
BestBuy is using Morpheus as the foundation of AI-based anomaly detection in their network.
GE Healthcare has built an echocardiogram that uses TensorRT to quickly identify different views of the wall motion of the heart,
and select the best ones for analysis.
Spotify has over 4 billion playlists.
They're using RAPIDS to analyze models more efficiently, which lets them refresh personalized playlists more often.
This keeps their content fresh with the latest trends.
WeChat is using TensorRT to achieve 8 millisecond latency
even when using the state-of-the-art natural language understanding models like BERT.
This gives them more accuracy and relevant results while maintaining real-time response.
It's great to see NVIDIA AI used to build these amazing AI services.
NVIDIA AI on EGX, with Aerial 5G:
the AI computing platform for the enterprise and edge.
The next wave of AI.