(upbeat music)
- Hi, I've Rangan Majumder,
Vice President of Search and AI at Microsoft.
Today, I'm really excited to talk to you about AI at Scale,
which is how we use large pre-trained language models
and how it's transforming our use of AI at Microsoft.
Every story about AI at Microsoft
starts with Bill Gates and Microsoft Research.
When he started Microsoft Research,
he had this vision that computers will one day see,
hear, talk, and understand human beings.
Over the last few years,
Microsoft Research has had many breakthroughs
from vision in 2016, where we're the first company
to reach human parity object recognition,
to speech a year later,
where we reached human parity
on one of the speech recognition challenges,
then machine reading comprehension,
we were able to actually read and answer questions
over documents, translation, and speech synthesis,
and then recently, last year, we were the first company
to beat the general language understanding benchmark
and got human parity on that.
It was a really tough benchmark,
which really needed us to understand language
across 10 different tasks.
So deep learning and Microsoft Research
has been really, really powerful
in terms of solving these extremely difficult challenges.
It's great that we're making so much progress
on these research benchmarks,
but what really matters is taking this technology
and bringing it to customers.
So what we noticed was
we had a lot of different AI groups across the company.
For example, in Microsoft Office,
we had a team working on AI
for a better reading and writing assistance in Word.
We had a team working on AI
for better search relevance in Bing.
We had a team working on AI
for better document management in SharePoint.
And then we also had a team in Outlook
working on AI for better email management.
We saw this big opportunity
where we can actually have these teams collaborate, share,
and actually build from each other,
rather than just working on these
AI algorithms independently.
So to accelerate the change that we needed,
we took advantage of three trends.
The first trend is transfer learning,
where you can train an AI algorithm on one set of data
and set of tasks, and actually reuse that same model
across a different set of tasks.
The second trend
is around large pre-trained self-supervised networks.
And what's happening here
is you can actually train a very large neural network
over massive amounts of data in a self-supervised way
where you don't actually have to give it
supervised labels from a judge or a human,
it can actually look at the data
and then try to train itself.
And with lots of data, we can train really massive models.
And these models are actually very, very powerful.
And then the last trend was a cultural shift,
where Satya says, "As a company,
we need to actually collaborate and share a lot more."
And the results have been phenomenal.
So if you think about where we were,
let's say three years ago,
the biggest model that Microsoft could train
was about 10 million parameters.
A year later, it was just 100 million parameters.
And then earlier this year, we trained one of the largest
natural language generation models at 17 billion parameters.
And with each order of magnitude number of parameters,
we see the model just get better and better.
So Turing-NLG is what we announced earlier this year.
And just recently, OpenAI actually trained the model,
there's 175 billion parameters on the Azure infrastructure,
some of it we were able to do.
So, let me show you an example of what's possible
with the Turing-NLG model.
The first is free form generation.
Here, I start and give it an example input,
like how was the universe created?
And this model actually just starts to generate
the text that comes afterwards.
So you can see once I give it the prompt
of how was the universe created,
it starts talking about the big bang theory,
and a lot of other scientific facts.
Notice that it's grammatically correct, it's fluent.
These are all the things that are ingrained in the model,
including the knowledge that the big bang theory
is one of the most accepted theories
around how the universe was created.
And this is all stuff
that it's not just copying and pasting from the web,
this is all knowledge that's ingrained
in the Turing-NLG model
from just reading lots and lots of documents.
For this AI transformation at Microsoft,
we had to actually rethink the entire AI stack,
from the very infrastructure,
where we had to get the best AI accelerators,
that includes Nvidia GPUs, Intel FPGAs.
The bandwidth, so we had to get really high
bandwidth network, where we got NVLink and InfiniBand,
so that these GPUs can talk to each other super fast.
And then software, where we actually innovated
and built a software called Deep Speed,
which allows us to train way bigger models,
much faster than if we didn't have it at all.
And this is all on top of the same infrastructure,
because what happens is these models are so big
they can't just fit on a single GPU,
you actually needs to spread them all out
across lots of GPUs.
And then the GPUs need to talk to each other.
So Deep Speed automatically does all that.
And then we have ONNX Runtime,
which can actually speed up both your training
and your inference.
So if you have this large model
and you wanna bring it to production,
you can actually use ONNX Runtime
for inference on the model.
And then finally, we had one centralized team
that would actually coordinate
across all their research groups and product groups,
and build these really large pre-trained models.
And that group was called the Turing Group,
which now builds these Turing NLP models for the company.
This allowed us to do a lot of custom capabilities
through reuse, no longer did we have each of these teams
working independently,
but we were to have one team
build these really massive NLP models, the Turing Group,
and then each of these groups would take that model
and fine tune them for their own scenarios.
So the Bing team can use the model
for their Bing scenarios,
SharePoint can use it for the SharePoint scenarios,
Word can use it for their Word scenarios, and so on.
And this is what we mean by AI at scale for everyone.
We had to build AI infrastructure at scale,
which includes the most powerful GPUs
that could run over tons of data
with really powerful networking.
We built really large AI models at scale.
So we're building models billions of parameters
that are super, super powerful.
And then we need to deploy these AI models at scale.
So we need to deploy them
across hundreds of millions of customers.
That means, you know, they need to be robust,
they need to actually run really, really fast
and they have to have low costs and high throughput.
Now let me show you another example
of what the Turing-NLG model can do.
Here I'm gonna show you summarization.
So what we've done is we've given a prompt
of, this is the summary of "Romeo and Juliet".
And then the model takes that text
and summarizes it in just a couple of sentences.
Now I'm gonna show you another example of summarization,
where we take a Microsoft article
about autonomous systems,
once again, we give it this really long piece of text.
And then the model is able to summarize it
in just a few sentences.
Feel free to try summarization on your own.
Let me show you some real examples
that you can try out in Microsoft products.
The first is Microsoft Bing
where we've created intelligent answers,
which summarize the web.
Here, you can ask a question like,
how much is a blood test for dog allergies?
And we read billions of documents in real time
to find the best answer,
which summarizes what's out there on the web for you.
Another example we have is semantic highlighting.
So you can go and search for define RNN.
And not only are we giving you documents,
which, you know, answers your question,
but we're actually able to highlight
the most important piece of text in that document.
So for example, define RNN,
we show a type of advanced artificial neural network
that involves directed cycles in memory.
This has been an awesome collaboration
across multiple teams at Microsoft,
which allow for faster innovation.
We hope it does the same for you as well.
For the least announcements, please look at the links below.
We're excited to see what you build
with the Turing language models and AI at Scale.
Thank you and goodbye.
(upbeat music)