So I got a lot of feedback on gpt3, a lot of people excited about GPT3 and the technology,
and a lot of people on the waiting list. I'm on the waiting list too. I signed up
early and have been on the waiting list for almost two months, and i've been waiting,
and waiting, and waiting and waiting, so i got sick and tired of waiting and decided to sign up
and download GPT2, install it on my computer. So i'm going to walk through how to get GPT2 set up
some of the pitfalls some of the things i learned in training a number of different models on gpt2
and some of the things to be careful of if you're starting a company using artificial
intelligence technology specifically gpt3 and how to be careful to make
sure that your company is set up for the long term. All right let's get into it.
So my previous videos i showed a number of different examples of gpt3 and i've gotten
a lot of comments of people wanting to try out those samples. Well a lot of those samples aren't
yet publicly available because GPT 3 itself isn't publicly available so you
can get on the waiting list and i've been on the waiting list for months and months and really
from what i've seen the waiting list has not made a ton of progress so GPT 2 was released a while
ago it's available on github so no problem. Download a virtual environment, anaconda.
Get that set up on your computer. Create a GPT 2 environment. Install the right version of python,
tensorflow, and gpt2 simple. Check out the code from github, download the 774 million parameter
model, it's just a couple gigs. Swear at your computer for a couple hours. Boom! You have
the shakespeare sample working. So now i can give my computer a prompt and it'll continue
to speak in Shakespeare. with the thought that I started. "So as am I, a smoke rises out of fire,
so smoke rises out of sweet smoke, so sweet it is. So sweet it is for me to win the love of
the sun to tear the dregs of the cheeks of old men." I didn't say it was good it just sounds
like Shakespeare now it's not particularly great. I don't happen to speak or have the need to speak
Shakespeare on a daily basis but it just shows the power of the technology and what's possible
even on your own machine. Now i'm on a MacBook and this was taking hours and hours and hours,
in fact it was taking so long that it really was starting to frustrate me and i spent an
entire day trying to figure out how to get virtual windows running under boot camp on
my Mac. Now this is because MacBooks aren't set up with the right graphics accelerator
most artificial intelligence is trained on a gpu and the mac just has a different gpu that's not
really trained for tensorflow and the technology i was using. I did find a better way to do this
and that's using google collab. So google collab is a fantastic technology it allows you to run
a gpu in the cloud using google servers, and so all those same samples that i was experimenting
with on my local computer for a library called gpt2-simple you're able to run in google collab
in the cloud. Google collab is a collaborative platform for running notebooks these are based
on a technology called jupiter. It's a combination of code and notes you can simply go through the
notebook either modifying the code or running it step by step. It's a really interactive and easy
way to learn artificial intelligence, learn how this technology works and run models right in
the cloud without having to update your computer or add a graphics card. So i'm using a notebook
from Max Wolf who wrote the library gb2-simple and it's a fantastic collab notebook that really
walks you through the steps of setting up and training and artificial intelligence using gpt2
i won't bore you with the code and the specifics but i've linked the notebook in the description
below so you can take a look at that and follow along step by step. You choose how big of a model
you want to train and there's a couple options for the size of the model that you want to train,
you download the open source model and then you fine-tune that model. So you can fine-tune it
by telling it what you want examples of. In my own experiments i trained three models to
get a sense of how it works the first model was trained on shakespeare which i showed you before,
the second model i trained on my twitter history and twitter makes it easy for you to download your
entire twitter history and by tuning gpt2 simple i was able to train it to tweet as if it was me.
Here's some of the best tweets that gpt2-simple was able to create. The internet is like a series
of tubes with a pipe that leads to nowhere, there is no escape. Killed another software project that
was on life support, sad? Nope. I love creating focus for the bigger things we have going on.
Some of the tweets worked some were confusing of course that's true of normal twitter too
the third example i tried to train GPT2 on a series of jokes. I was trying to get it
to be funny so i downloaded an entire library of jokes and tried to fine-tune gpt2 to tell jokes.
Now i was able to get gpt2 to have a really good structure of jokes and so the format of what it
was producing was structured to be like a joke but many of the jokes that produced weren't actually
that funny. So hopefully this gives you some of the bare bones steps you need to get started
with gpt2 now if you're thinking about using artificial intelligence for your startup there's
some caveats, some things to be aware of. The first is ai is often about the data and in my own
experimentation whether it was the twitter history or the shakespeare archive or the joke database,
the database of the data that you're feeding and training your ai model is really important.
Really critical to getting good results and fine-tuning that data, fine-tuning how you're
training that artificial intelligence is really what separates a great artificial intelligence
model and one that's really okay. So really plan on spending a lot of time curating your
examples curating your samples to get better results with your artificial intelligence model
the second thing to think about when starting a business using artificial intelligence
is your dependencies. Are you dependent on open ai and an api that they're providing and
what happens if that api stops functioning or functions differently. I happen to think that
using artificial intelligence can be a great differentiator for a startup but if you're using
the same api that every single other developer is using for their model you're not really creating
an opportunity for clear differentiation. So think about how you will differentiate your
startup using an open api versus something that's proprietary and special to your startup.
You don't want to have a dependency on a third party that prevents you from really being
competitive. In differentiating yourself i've seen a lot of pitches from startups that use artificial
intelligence and it's important to remember that the business side of an artificial intelligence
company is as important if not more important than the artificial intelligence itself think
about the business that you're trying to establish and how that business channel will really function
where are you going to get your customers. How you're going to differentiate yourself and long
term as more companies incorporate artificial intelligence. What's going to set you apart
in terms of what you're doing from a business perspective from your competition. And lastly
gpt2 and gpt3 are hot technologies today lots of people are talking about them but you can be sure
that google and amazon and apple and facebook and many other companies will create technologies
in the future to compete with gpt 2 and 3 and provide technologies for application developers.
So think about long term how you plan to pivot and incorporate or use new technologies
by creating strong dependencies with one provider you could actually limit your long-term potential
for your startup or business. So if you're a startup using artificial intelligence technology
or gpt-2 or gpt-3 i'd love to hear from you leave a comment down below telling me how you're
using the technology and how you think you'll differentiate and if you're new to the channel,
I talk about entrepreneurship, technology and design and i hope i'll see you in the next one.