Our ability to communicate,
to express ourselves through written, spoken, and visual language
is a fundamental aspect of what makes us human.
If we want to create an artificial intelligence capable of augmenting human ingenuity
we must start with language.
Innovation hinges on our ability to see the world differently,
breaking boundaries, and looking in unexpected places
to help solve some of the world's toughest challenges.
[The Future from an Alternative Perspective]
[A L T]
As humans we master language by making connections.
It takes us many years to learn the relationship between words
and the importance of context.
I'm a new father to a one-year-old girl and I see her learn little by little about language.
Every night we read her books with pictures of animals, things, and places;
my wife in English and I in Spanish.
Basically, we're giving her large amounts of language and image data in two languages
and I can see her starting to correlate concepts across the two.
The other day she saw a puppy in the street, and when my wife asked her what it was
she said, "Dog". When I asked her she said, "Perro".
There is hidden complexity in seeing an object and then identifying and accessing the word
or words needed to describe it, and then doing this in multiple languages.
Now, imagine what it might take to understand a sentence,
a whole Wikipedia article, or an entire novel.
In the past we used rule-based systems to try to mimic this understanding in machines,
but there was not genuine language comprehension at all.
Later, we taught these machines how to model language in a mathematical way with vectors.
Still, this didn't really give our machines a rich understanding of conceptual relationships.
What was missing was context.
Today our language models are learning language by essentially reading the book
through an approach called self-supervised learning.
It works like this:
We give our model a huge amount of text and as it processes the text
we ask it to predict words based on the surrounding context.
As the model does this hundreds, to millions, to billions of times
it starts to build an understanding of the underlying language concept and grammar.
We think of the development of AI similar to how we think of the development of the human brain.
Synapses between neurons build cognitive capabilities in humans
and in AI these neural connections are called parameters.
A year ago the largest AI models had around one billion parameters.
Our latest model called Turing has seventeen billion.
We anticipate that the trend would continue into the hundreds of billions of parameters
and then trillions.
The only way to train a model with billions or even trillions of parameters
is with large scale infrastructure composed of clusters of hundreds
to thousands of machines with specialized hardware
are interconnected by high bandwidth networks.
The Azure cloud provides us with such clusters.
We then need to distribute the training of the model across all of these machines.
Over the past year we have developed optimization techniques
to reduce the resources that we need and still train models faster
on the same infrastructure.
We have recently open sourced these techniques to enable everyone to train large AI models
much more efficiently.
General language is sometimes not enough to support the tasks needed.
Let's say you give a human a foundational education where they gain a general understanding of the world.
This is similar to how we develop our AI models.
We start with a high school level model which has learned from all the data available on the internet.
We send this baseline model to build on its abilities in specific domains.
For example, the productivity domain for Office, the business domain for Dynamics,
or the professional domain for LinkedIn.
As our language model learns from domain data, it does so within the context of each organization
in a privacy compliant manner.
An emerging generation of even larger models are now learning from this data
and functioning in a more generalized way, combining language, images, video, and speech.
Even with these amazing recent achievements, the reality is AI is still in its infancy.
For example, we're still far from from AI that can use common sense reasoning to draw conclusions.
In a search for greater intelligence, civilizations and humanity thrive.
Every time we reach a new AI frontier we realize we can do better.
As research advances and we magnify human intelligence with artificial intelligence,
we may gain a clearer and deeper understanding of ourselves and of one another.
[The Future from an Alternative Perspective]
[A L T]
[microsoft.com/innovation]