>> Are you rolling already?
Is this part of the show what I'm doing right now?
[MUSIC]
Hello and welcome to this episode of the AI show.
My name is Seth Juarez, and we're going to talk all
about machine learning models,
what they are, how to make them,
and then what the process is to actually make them work.
So let me start with a couple of
basic concepts and we'll go through the super
fast because I want you to
understand what's going on because I had a hard time with it.
I was a programmer and then to data science afterwards.
So generally, as a programmer when you solve a problem,
what you do is,
you make a function,
you get some input,
and out comes some answers.
That's basically what we do as programmers,
but it turns out that there are certain problems that are
really hard to solve. I'll give you an example.
If I give you the bits of
a picture and then I ask you what's in that picture,
that's a hard problem to solve.
As a programmer, you may be able to solve it.
If you're given enough examples,
there's enough for-loops, there's enough if statements,
that you have to go home and take a shower afterwards when you
write that kind of code, totally understand that.
But in this case,
the space of problems where it's hard to
program but the outcome is pretty simple to understand,
that's where machine learning does really well.
What it actually does,
is it flips the whole paradigm
from instead of you writing a function,
you give the computer answers and input,
and it will make a function,
which is pretty cool.
That's what machine learning is all about.
Specifically, machine-learning answer a set of questions.
One we're going to answer today is called Classification.
Basically, give it a thing and
it will classify it into specific band.
That's the one we're going to solve today super quickly.
Now, the one thing though that's important is that this,
we call it functions and input.
Data scientists call the thing that
goes in the answers and the input,
data, and then the output is called a model.
So if you've ever heard of a term called machine-learning model,
now you know that it's basically
a function that's written in an alternative way.
Basically, when we write them,
we are basically telling it
the structure and how to
learn it. We're going to go through that today.
So again, a machine learning model is basically a function.
I'm going to let that sit with you for
awhile because sometimes we make it into more than what it is,
but basically we're learning a function.
Instead of you having to write
a crazy function with a lot of if statements,
for loops, whatever, the computer
will figure out the function right now.
It's pretty easy to understand what the x is.
We're going to do a simple problem here,
so that everyone can understand what's going on
because it's the best way that I understand the thing.
So the x in this case is this.
Now, if you squint hard enough into the matrix,
which is what this is,
you will see a number and notice that that is the number zero.
It's basically a grayscale image that has a 28 by 28 pixel number.
That's the input.
Basically, anything you do in
machine learning and in particular for classification,
you have to find a way to convert the things that go
into this function that it's going to invent into numbers.
If you have to do that with text,
if you have to do that with pictures,
you have to do that with sound,
and notice that with features it's pretty easy
because we have a color picture,
you have red, green, blue,
the three channels, and then height and width.
For text is a little bit different.
You have to do some things like, for example,
bag-of-words models or TFIDF,
n-gram models, or another way to convert the text into numbers.
If you're doing something like speech,
notice that the wave is basically a height over a period of time.
You can do a number of things like that.
For example, fast Fourier transforms will separate
all the different frequencies and
so you can learn off of those as well.
Basically, you have to convert things over to numbers.
Now, the input's easy.
Once the numbers are there, the question is,
what does the function look like?
That's the important part.
It turns out that with machine learning,
it's not just going to invent a function out of whole cloth.
You have to give it a structure in which to learn.
For example, if you were to give it
only if statements to work with,
you would make a decision tree.
If you were to give it a bunch of
other structures and it would learn other things,
the framework that's used like
scikit-learn, TensorFlow, and PyTorch,
there are a number of options for selecting the function shape.
First, I'm calling a function shape,
data scientists call it model.
So let's do this in TensorFlow.
I like TensorFlow, it's an amazing framework.
There's others like PyTorch,
scikit-learn like I said before,
among a bunch of others,
you can choose whatever you want, but
the concept is still going to be the same.
You're going to make a model
and there's a specific way to make it.
In TensorFlow 2.0, there's a couple of concepts.
The first is, you have to craft the function shape.
You have to do that in TensorFlow.
You have to do that in PyTorch.
Scikit-learn has a bunch of pre-packaged ones
that then you can customize. That's the first thing.
The second thing you have to do is you optimize the model
using a cost/loss function and an optimizer.
Now again, the model is the function that you're going to execute.
The cost and loss function tells you how bad the model is added.
The optimizer's job is to take that bad function and say,
"We want to make you as less bad as possible."
It does that in conjunction with a mouse.
So those are the three things you need.
You need the model which is the function we're going to learn,
a cost and a loss function which says
how bad is the function that you learned,
and then an optimizer which basically takes the loss function,
which has the model function in it,
and it optimizes that for greatest benefit.
Now, once you have the model,
you use the model to make predictions because again,
it's basically a function.
The output is just a file like a JAR file or an assembly file.
The model is just a function that you call.
It has a funny-looking input.
Like we said, it has to be only numbers.
The output can sometimes be a little funny,
but that's basically what it is. All right.
So let's talk about these concepts
a little bit more so that you can get
a sense for them. The first is the model.
That's the h of x function that we
want to figure out its any function.
Remember that we wanted to see what was in
a picture and it was hard to write that function.
So any function like that that's hard to write,
that's the model function that we're going to use to execute.
The second is the cost and loss function.
Now, here's the interesting thing and
I hinted at that it's contained.
But notice that the loss function uses the model function and y,
which is the right answer.
Remember how I told you at the beginning?
You had to give it the answer
and the input, and then it could learn.
The answer is used in
the loss function to measure how bad we are at it.
We want the value of that function to
be zero because that means we are zero bad.
The higher that number,
the more bad your model function is.
Then the optimizer takes that loss function that you see up there,
and it tries to minimize it using a bunch of certain techniques.
Now, here's the thing.
This is where the craziness ensues.
Number one, you have to craft the function. You have to do it.
There's a couple of
models that have been built that are really good.
I've talked to a world renowned experts, and they say,
"You basically start with the shape that people have
already agreed upon and that seems to
work and then you optimize from there."
So you have to pick that.
So notice that the machine
isn't learning stuff out of whole a cloth.
You have to give it a direction in which the learning,
you do that through of crafting the model function.
The second is you optimize the model
using the cost loss function and the optimizer.
That's how you do it. We're using TensorFlow to do that.
Then finally, use the model to predict.
Okay. So let's take a look at what this looks like in TensorFlow,
if you want to see this.
You craft the function shape.
Notice that the model there,
I called it model,
has a sequential shape which basically has
a bunch of layers that all the numbers flow through,
and then out comes an answer.
In this case for the number problem,
the answer will be a vector of size 10 and in each of
the little buckets there will be
a probability of which one it thinks it is.
Then, the input is 784 numbers that
represent the intensity or the grayscale image, basically.
That's all that's happening.
Now, the second thing you do is you take the model
and you optimize it using a cost function and optimizer.
I'm repeating these things a lot because it took
me a lot of time to get into my head.
In this case, we're using the Adam Optimizer.
The loss function we're using is something
called sparse categorical crossentropy,
and the metric we want to optimize is accuracy.
Finally, you'll use fit.
Fit, what it does, is it takes the model function,
the cost function, and the optimizer,
and then it goes through this giant for-loop of trying to
optimize all the parameters inside of the function shape.
Then you got use the model to predict.
Now, here's the thing.
These are all the optimizers that you can choose from.
These are all the loss functions you can choose from.
Function shapes can be anything, like here's three.
Notice that you can make them into whatever shape you want.
So then the question would come up,
how do I know what function shape to choose,
number one, what loss function to use,
number two, and then, what optimizer?
Well, this is why it's called data science.
Because in order to answer the question properly,
you have to use science,
and science tells you that we have a question.
We come up with an idea of the function shape, well,
loss function and optimizer,
we test it out,
then we analyze the report.
This here loop is called the Data Science loop of sadness,
where you're going to try a bunch of things and it
may or may not work. That's okay.
It will [inaudible]. So if you've ever looked at data science code,
the code is not written to be put into production.
The code is used to prove whether a model works or not.
So that's why the code is a little bit different.
So I think I said a lot of stuff.
Let's go and take a look at
my particular implementation of the digits one you saw.
This is a model.
This model, if you want to look it up,
it's called a Convolutional Neural Network,
because it uses these things convolutions.
In summary, it basically finds a way to
make dumber images to learn from.
That's the way I like to think about it.
So here is the actual model function.
Here is the compilation of the model to use the Adam optimizer,
the sparse categorical crossentropy.
Then you can see here that I call fit.
What fit does, is it uses
those three things to optimize. Here's the thing.
You may get terrible accuracy,
and that literally means that you chose
the wrong model or you chose the wrong loss function,
or you chose the wrong optimizer.
So you have to go through and solve for those things.
So let me run this and show you what
this wonderful for loop looks like.
So I'm just going to hit "Enter" here.
Then, I'm going to warm myself by the fire of the GPU.
Because as this run,
it again takes a little bit of time, and that's okay.
So you'll see it run behind me.
It's going to take some time because basically,
it's trying to figure out,
given that function shape,
given that loss function,
and given that optimizer,
what are the best parameters that it can learn for that model.
Inside each of those models,
there's hidden parameters that it uses to figure these things out.
Basically, they're numbers in a network.
You can see here that I'm going to do
this go over it five times in
my dataset and the accuracy numbers are there. It's pretty cool.
Now, it's going to take a little bit of time to run.
But what I'm going to do is I'm going to show you
what a model function actually looks like.
I think you're going to be a little surprised
because it's not any magical thing.
All right. So I'm going to lower this here.
What I'm going to do is I'm going to
open this model function right here.
You are going to see that just like I said,
it's basically a network.
By the way, the program we're using here is called Netron.
When I save the model out to Onyx
or to an HDF5 which is a Keras file,
basically, it'll load it up.
If you look inside of the actual AI model,
there's no flying unicorns or anything.
Basically, it's just numbers.
These numbers, as you hear my machine cranking through,
are learned through in this iterative process.
It may not work the first time,
it may not work the second time,
but the idea is that it's going to learn the right thing.
So we are almost done with epoch number five.
In this case, you can see that it gets 99 percent accuracy.
It saved the file here to latest HDF5.
That's the file my friends,
that's the artifact of the machine learning process.
Okay. We've spent about 13 minutes
going over what machine learning models are.
If you'd like to learn more, go to aka.ms/MachineLearningModels,
and I'll have much better documentation there and things that you
can do to optimize your own models.
Thank you so much for watching. We'll see you next time.
My name is Seth Juarez. This is the AI Show. Take care.
[MUSIC]