(upbeat music)
- There have been a lot of news about ChatGPT lately
like people using ChatGPT to write essays,
ChatGPT hitting a hundred million users,
Google launching Bard to compete against ChatGPT
and Microsoft integrating ChatGPT
into all their products, and also the viral sensation
of CatGPT where it can answer all of your queries,
but as a cat, meow, meow, meow, meow, meow, meow.
ChatGPT, if you don't know already, it's a chat bot
by OpenAI where you can ask it many things.
For example, explaining complex topics
like explain why I'm a disappointment to my parents
or ask it more technical questions like,
how do I inherit more money than my brother from my parents?
A lot of people are using it to write essays, draft emails,
and even write code.
So I tried it myself, of course, as a YouTuber obviously,
my first question to it was, who is Joma Tech?
And it answered...
Are you fucking--
You know, ChatGPT has a lot of limitations,
like here we ask it to name colors
that don't have the letter E in them,
and this is what they gave us.
Orang, yllow, red, that's clearly wrong.
In all seriousness,
this is to demonstrate how ChatGPT works.
It's a pre-trained large language model,
meaning it was trained on text data
from the internet until the end of 2021.
So it won't know anything
about things that happened recently.
It doesn't have access to the internet.
It'll only predict the answer based
on what it has consumed already,
and the way it answers your question is
by predicting each word that comes next.
For example, if you ask GPT who Bard is,
it's not going to know.
You might ask Joma, didn't your channel launch in 2017
and ChatGPT was trained on internet data until 2021,
yet it doesn't know who you are?
Yeah, so there's actually a technical reason
and fuck you.
Recently ChatGPT hit a hundred million users.
It launched November 30th, 2022,
and this article came out February 3rd, 2023.
So it took two months to hit a hundred million users.
Who are these users and what are they doing with ChatGPT?
Well, it's pretty obvious, they're cheating with it.
Everybody's cheating such that
some school districts have banned access to ChatGPT.
If they can write essays, then they can pass exams.
ChatGPT was able to pass exams from law school,
business school, and medical school.
Three prestigious industries.
Now, this is why I went into coding
because I always thought that law school,
business school, and medical school,
it was too much about memorization
and you're bound to get replaced,
it just wasn't intellectual enough, you know?
All right, well,
I guess engineering is getting replaced, too.
ChatGPT passes Google coding interview,
which is known to be hard, but I guess not.
But note that it is for a L3 engineer,
which means it's a entry level, for those not in tech,
there's no L2 and L1, it starts at L3,
but this does raise questions about ChatGPT's ability
to change engineering jobs behind it,
and we're already seeing the change
as Amazon employees are already using ChatGPT
for coding even though that immediately after,
they told them to stop, warning them not
to share confidential information with ChatGPT.
What's happening is they're feeding ChatGPT
internal documents, which are confidential,
but OpenAI stores all that data.
You know, it reminds me of when I used to intern
at Microsoft and they didn't let us use Google
for searches because they think that they might spy on us.
I was like, relax, I'm an intern.
I'm not working on anything important.
In fact, I actually wasn't working at all.
You know, I was playing Overwatch all day,
but yeah, anyways, they forced us to use Bing for searches.
One thing that's being underreported
in mainstream media is the success of GitHub Copilot.
It's probably the most useful
and most well executed AI product currently out there.
Have I used it?
No, I haven't coded in forever.
Now, here's how it works.
The moment you write your code,
it's like auto complete on steroids, like this example,
it helps you write the whole drawScatterplot function
and it knows how to use a D3 library correctly.
Another example here, you can write a comment
explaining what you want your function to do
and it'll write the code for you.
Sometimes even the name
of the function will give it enough information
to write the rest of the code for you.
It's very powerful
because it's able to take your whole code base as context
and with that, make more accurate predictions that way.
For example, if you're building a trading bot
and you write the function get_tech_stock_prices,
it'll suggest, hey, I know you're going
through a rough time,
but building a trading bot is not going
to fix your insecurities and maybe you should just accept
that you'll be a disappointment for the rest of your life.
Okay.
How did all of this happen?
Why is AI so good suddenly?
The answer is the transformer model
which caused a paradigm shift
on how we build large language models, LLM.
By the way, this diagram means nothing to me.
It makes me look smart, so that's why I put it on there.
Before transformers,
the best natural language processing system used RNN,
and then it used LSTM,
but then Google Brain published a paper
in 2017 called "Attention is All You Need"
which is also my life's motto because I'm a narcissist.
The paper proposes a simple neural network model
they call transformer, which is based
on the self attention mechanism
which I don't fully understand, so I'll pretend
like I don't have time to explain it
but I also know that it allows for more parallelization
which means you can throw more hardware,
more GPUs to make your training go faster
and that's when things got crazy.
They kept adding more data and also added more parameters
and the model just got better.
So what did we do?
We made bigger models with more parameters
and shoved it a shit ton of data.
Sorry, I'm trying my best here to make the model bigger.
All right, fuck it.
Anyway, that gave us ready
to use pre-trained transformer models like Google's Bert,
and OpenAI's GPT, generative pre-trained transformers.
They crawled the whole web to get text data
from Wikipedia and Reddit.
This graph shows you how many parameters each model has.
So as you can see, we've been increasing the number
of parameters exponentially.
So OpenAI kept improving their GPT model
like how Goku kept becoming stronger each time
he reached a new Super Saiyan form.
While editing this,
I realized how unhelpful the "Dragon Ball" analogy was.
So I want to try again.
To recap, transformer was the model architecture,
a type of neural network.
Other types of models would be like RNN and LSTM.
Compared to RNN, transformers don't need
to process words one by one,
so it's way more efficient at training with lots of data.
OpenAI used the transformer model and pre-trained it
by feeding it a bunch of data from the internet
and they called that pre-trained model GPT-1.
Back then, NLP models would be trained from scratch
for a specific task like translation or summarization.
Both transformer, we get to pre-train the model first
then fine tune it for a specific task.
Then for GPT-2, they did the same thing, but more
and with a bigger model, hence with 1.5 billion parameters,
and then with GPT-3,
they went crazy and gave it 175 billion parameters.
However, just like raising a kid,
just shoving it with a bunch
of information unsupervised might not be the best way
to raise a kid.
She might know a lot of things,
but she hasn't learned proper values from her parents.
So that's why we have to fine tune it, tell it what's right,
and what's wrong, how not to be racist and clean up its act.
That's GPT-3.5, a more fine-tuned version of GPT-3
with guardrails that can be released to the public.
Now you have a decently well-behaved kid,
but you now want to show her off, so you dress it up nicely,
get her ready for her first job, AKA more fine tuning
with some supervised training
so it behaves properly as a chat bot.
That way it's well packaged and is ready to ship
to the world with a web UI.
Okay, back to the original shitty "Dragon Ball" explanation.
So you can think of Goku's hair,
like the number of parameters, 175 billion parameters,
which is why you can see Goku has more hair now.
Goku hair isn't much longer,
but it's just styled a little bit differently.
100 trillion parameters.
So technically GPT-3 was already amazing
but OpenAI was able to package it neatly with ChatGPT
which made it user friendly, so it became a viral sensation.
So yeah, packaging is important.
It caused everyone to really pay attention to this.
So how did people react to the viral growth of ChatGPT?
People were mind blown and said, Google is done
because ChatGPT is going to replace search engines.
No, it can't.
Until it can search for porn,
it cannot replace search engines.
Oh, wait, why search for porn
when you could generate it?
(upbeat music)
Anyway, even losing a bit of search volume
to ChatGPT would be a big deal for Google
since 80% of their revenue comes from ads
and most of it comes from search.
People were telling Google to release something similar.
Google was like, bruh, we have LaMDA,
which is basically ChatGPT, but releasing it would be risky
as they had much more reputational risk at stake
and has to move more conservatively than a startup would.
That's foreshadowing by the way.
Microsoft is chilling.
They positioned themselves really well
by investing $1 billion in OpenAI early on in 2019.
That allowed OpenAI to leverage Microsoft's Azure
for its compute power to train and run their models
and Microsoft gets to integrate OpenAI's tech
into their products.
So if OpenAI succeeds,
Microsoft succeeds and remember GitHub Copilot?
Well, GitHub is owned by Microsoft, so that's a huge win.
Meanwhile, Google is panicking
and issued a code red,
calling in the OG founders Page and Brin.
Actually I have no idea who's who, so...
Anyways, but they called them to strategize
on how to approach this.
Microsoft is fueling the momentum, especially
with ChatGPT growing so fast
and the tech is very promising.
So Microsoft invests another $10 billion
into OpenAI for a 49% stake in the company.
That money can help OpenAI,
I don't know, unlock Super Saiyan 4, maybe.
Microsoft also plans to integrate GPT
into Microsoft Teams following the same playbook
as what they did with GitHub Copilot
which would be huge for them.
Google also made some additional moves.
Google invests almost $400 million
in OpenAI's rival Anthropic, which is pocket change compared
to the $10 billion Microsoft invested.
If you don't know what Anthropic is, it doesn't matter.
It's like the Burger King of OpenAI.
Google goes back on their word
about not launching a ChatGPT clone
and announces Bard AI, a ChatGPT clone.
Remember when I said they didn't wanna launch
a ChatGPT competitor because of reputational risk?
Well, funny enough, that's exactly what happened.
The AI made a mistake in the ad
and Google shares tanked, losing a hundred billion dollars
and I still own my Google stocks from when I worked there.
The mistake was Bard said,
"JWST took the very first pictures
"of a planet outside of our own solar system."
But this astronaut said, "No, it was not true, Chauvin did."
That tweet alone cost me a lot of money.
Anyway, Microsoft responded to the announcement
by releasing a new Bing with ChatGPT built in
to compete with Google search.
Meanwhile, we have Meta, who is in denial.
Meta's AI chief says,
"ChatGPT Tech is not particularly innovative."
That is just massive copium.
Finally, we got Netflix,
who's too busy cracking down on password sharing
to care about AI.
All right, what about us engineers?
What's the future for us?
The reality is
that GPT isn't replacing anybody's job completely.
Like most technological innovations,
that change can seem drastic
because the media loves dramatic titles.
But if you're open-minded, you have time to learn
about it and embrace it rather than fighting it.
If you're a software engineer and you feel threatened
by ChatGPT being able to solve FizzBuzz, oof,
then you should maybe consider becoming a YouTuber.
Just kidding.
Please don't compete with me.
Though, you should incorporate ChatGPT
and GitHub Copilot to your workflow.
It really removes tedious parts of software engineering.
If you're working in a new language or API library,
you don't have to Google, sorry, Google,
you don't have to Google endlessly
for the stuff you already know.
Just break down and describe your problem
to ChatGPT to get a huge headstart
or get good at coding alongside Copilot.
If you structure your code base well
and write good comments that describe what you want to do,
Copilot often gets the logic problems right.
It's a symbiotic relationship.
Become the cyborg.
See, the trick here is that, as a software engineer,
your job is to translate
and break down a business problem into software problems.
Your job is to know what questions to ask
and what answers to accept.
In fact, here's my prediction.
GitHub Copilot is not done innovating here.
Their next big product release will turn an issue
or PR description into an actual full-blown code commit.
So as a software engineer in 2024, you better get real good
at writing GitHub issues and reviewing PRs.
All right, that's it for this ChatGPT video,
but I think this ChatGPT narrative is just one battle
of a bigger AI war that's happening
between Microsoft and Google.
I'll talk about that next time.
See you and thanks for watching,
and remember to call your parents.
(upbeat musical effect)