demo of some of the technologies and some of the things that GPT 3 is able to
do and some of the demos circulating the
internet in just the first few days and how this technology works and what
it means for the future of technology, software and how we use the internet
going forward.
So GPT-3 comes from a company called OpenAI.
So OpenAI was founded by Elon Musk and Sam Altman.
Elon Musk is a prolific entrepreneur you maybe familiar with him from SpaceX and
Tesla, and Sam Altman was one of the founders
of Y Combinator, a very famous startup accelerator. Combined they
invested over a billion dollars in OpenAI to advance the state of the
art of artificial intelligence and make sure that artificial
intelligence was being used for the betterment of mankind.
So let's jump in and take a look at what the tech can do.
So the first example is an application called Figma and the developer has built
a plug-in to Figma that allows users to enter a description
of what they want for an application. So in this example, a user is describing an
application with a feed and some icons and a user interface and
when he hit submit the application actually
generates an application that looks very much like Instagram. it has all of the
buttons all the UI in a scrolling feed. Just astounding in terms of what
it's able to do. The next example is a coding example and
so this is Debuild.co and they're describing an application
that is a basic to-do list application. And so what the software is
doing is it's generating a React application. All
of the functions and procedures, all the things that are necessary to create a
React component to handle to-dos. And you can see it works.
You can add items to the to-do list there's all the events and triggers for
handling the interactions of the application
and it's all generated on the fly by artificial intelligence.
Just amazing what we're able to do with AI both
design and development aspects. The next example
is in Excel, and so we have a table with headers of company ticker and year
founded. And we ask OpenAI to complete the table
and it's able to fill in the entire table with the data that it thinks we
want. Again it's pulling this information from
its corpus of artificial intelligence and it's trying to be predictive in
terms of what it thinks we want in the table.
If we don't agree or we want to train it a little bit differently we can cue it
with different elements that start in the
table. So here I actually want the table to be filled in starting from
the newest to the oldest and so I put twitter and
facebook, and then ask it to predict the table.
And it's able to understand what I'm trying to do, and again goes
top to bottom all the way in, filling in the year founded information with all
the company information. So this example is within Wikipedia. So
if you've ever seen a long article you can't really understand
or find the information that you need, you can simply have a plugin where you
ask the exact question that you want. So in this example, I'm asking why is bread
fluffy and it's searching the data and information
in that article and generating an answer to my question and giving me a citation
and location that i can jump to. So really summarizing the data making it
really easy for me to find exactly what i want from large volumes
of text and information that are presented.
Because of the vast amounts of information within GPT 3 you can really
ask it anything. In fact you can almost use it like a
search engine, where you ask it a question it can give
you one result giving you the citation of exactly where to find that
answer. So Google, and Apple and Amazon have thousands of engineers working on
this problem and we're able to use this artificial
intelligence to solve a very similar problem
almost instantly. Just amazing what you can do with the tech.
One of the common applications of GPT-3 is
translation. So here we can give it an example of an English sentence that we
want translated. "Your order will arrive in seven to ten
business days" and ask it to translate into French.
We can also use translation to translate things that are written in legalese. This
is an NDA, a non-disclosure agreement and
translate that into something that a second grader
would understand. Really useful. So this is the paper from
OpenAI it's called language models for few-shot
learners. Now the idea behind the technology
is that it can use the models in the text to train things on.
Zero shot, one shot, and few-shot models. And so
the example of a zero shot might be to translate
English to French and you give it a prompt like the word cheese and then it
translates that word. As the model is being fine-tuned it gets better and
better at zero shot. The more examples of text you give it
the better it can get. So here's an example of one
shot where you give it one example, and here's an example of a few shot model
where you give it a couple examples of what you're looking for.
And it's able to do the translation for you.
As you train more, the artificial intelligence is able to
predict what you're looking for in terms of an answer,
in terms of more and more models. As you train the model for more and more days
the software starts to converge on what the answers are. And so this is the
petaflop days as it's computing. So here's an example
of one of the recent demos where people can speak in plain english
and and the artificial intelligence is able
to translate that into command line tools that are needed
to run a linux operating system. So instead of
an esoteric command, you can just say things like "list files recursively"
and it knows exactly what to do or "list files in a human readable format"
and it's able to return those results as well.
There's lots of other applications for the technology you can use the
artificial intelligence to generate quizzes
based on the information that you have. So it can generate vocabulary or questions and
then show you the context of where it's
getting the information for the quiz that it's generating.
It can also be used for entertainment such as role-playing and generating a
story with multiple endings or multiple directions in which you can go.
A really early exploration of what the technology is capable of but
gets your imagination going in terms of what's possible.
OpenAI has been developing its technology for a number of years and one
of the very first papers was on generative pre-training. The whole idea
of generative pre-training is that most artificial intelligence is trained
on labeled data. And so you have a large
collection of data and the data is manually
labeled so that the artificial intelligence knows exactly that the data
is correct, and what it means. And what genitive pre-training means is
that they realized that there's a ton of data
out there that hasn't been labeled. And in particular data like the entire
internet or Wikipedia or books. And if they could
train the artificial intelligence to essentially label itself, and learn from
that existing data you could really accelerate the pace of
artificial intelligence. The first generative pre-training model
used about 7000 books in its collection and the newest generation the gpt-3
uses a much larger collection of data. How much more?
Over 410 billion parameters taken from the web
67 billion parameters taken from books, 175 billion parameters in total and much,
much, more. That's the sheer mountain of data that's
being used to train this artificial intelligence.
Because the data model is pre-trained the model gets better and better the
longer you run it, and the openai team ran it for over a
thousand petaflop days. So the data was trained on
over an exaflop of computing power. And just
to imagine how much computing power that is...
One second of exaflop computing power is equal to 37 trillion years
of adding numbers together. So it's an an absolutely staggering amount of computer
horsepower. I'm really excited about the future of
artificial intelligence i think that technologies like GPT3 and future
iterations of this will change how we interact with
computers. This technology makes things like Amazon Alexa, and Google
Assistant and Siri look like real child's toys,
because of the sophistication of what this technology can do.
i'm incredibly excited by artificial intelligence i think this is going to be
transformative. i hope you enjoy this type of content
I'm Greg Raiz. I talk about technology, entrepreneurship, and design and i hope
i'll see you on the next one.