>> You're not going to want to miss the next couple of episodes
of the AI show where I take you
inside a talk I did at the Toronto AI user group.
These bite-size chunks will talk all about what AI is,
how Machine Learning works,
and maybe even a little bit about Deep Learning.
Get into it, understand it, make sure you tune in.
[MUSIC]
How's it going everybody? I am Seth.
Our org is different than all the other ones
because I am not here to sell products,
I'm here to solve problems.
If you like the alliteration on that, solve problems.
So basically, this is an introduction to deep learning.
But my sense is that all of you know a lot of stuff,
so I will go faster.
So depending on what's going on,
because I have put way too much stuff in here.
Basically, I'm going to motivate
Convolutional Neural Networks from first principles,
and then show you what it looks like in
TensorFlow because you probably all know what it looks like,
and then show you how to run it on Azure, basically.
So obviously, that sounds like a lot of stuff because it
is and so I will go
fast or slow depending on what you tell me to do.
So since Canadians are super nice,
you can basically be, just be like, okay.
Or be like, and I'll know,
and you're nice way and you won't have to say
sorry for it afterwards, just FYI.
So the thing about machine learning and deep learning,
and since you are all,
I'm assuming practitioners, and
if you're a beginner, that's okay too.
This isn't for everybody.
The thing with machine learning is we have
this super huge gap of
understanding to overcome because
terrible things have been done with AI and machine learning.
Come on, that's funny. Is it clippy?
They're not, they're all looking at me.
These are the jokes people.
The thing about it is, is that everyone today is
all about machine learning or deep learning.
As you know, the difference between machine learning and
deep learning depends on the marketing guy, effectively.
So that's the important thing.
The reality of the matter is when I explain it to people,
let's just say because I'm assuming you-all know this stuff,
but this is how I explain it to
people so that if you ever need to.
I always say, "Look,
when we're talking about AI, it means a lot of other things too."
If you remember, like way back in the day
when Deep Blue chess thing beat Kasparov,
that's AI, but it's not machine learning.
It's very important to distinguish.
The way I like to think about it is the search space.
The size of the search space helps you know where things are.
The search space in chess,
while large, is still finite.
The search space in the amount of
pictures that people generate is infinite.
Because I have seen some crazy stuff on Twitter and you know this.
Then our parents put really crazy pictures on
Facebook. So what does it originate?
Artificial intelligence and machine learning.
In my professional opinion,
Artificial Intelligence is a glorified search problem.
Remember algorithms like A-star, Minimax,
they're all searching entries with a good strategy.
This is how I like to think of AI.
I like to think of it as the hard part of
AI where there is a fixed problem space.
When it comes to machine learning,
there is a little bit of squishiness about the problem.
Instead of searching a space,
what we're doing instead is providing
examples to generate an algorithm.
Most people don't like this.
Because when I say algorithm, I mean model,
we're building a machine learning model.
But if you think about it, it's
just a different way of making a function.
It just looks weirder.
If you open like an HDF5 file
in front of people that have never done machine learning,
they think the machine's going to
explode and glitter is going to come out.
Because it's AI, but it's the file, and I say, "Well,
it's basically a bag of numbers and a network, sorry."
Sorry to disappoint you Steve for marketing,
it's not anymore than that.
It's a function that's written with data.
So then wasn't originally machine learning and deep learning?
Because some will tell you that
neural networks belong in the space of machine learning.
But then if you get to n number of layers,
then it becomes deep learning.
It is still an open question what that number n is.
Again, it depends on the marketing people, in my opinion.
Because that you've seen resonates,
I think how high they go,
they go pretty high now.
Reason, I'm a computer vision guy,
and so I think in terms of computer vision,
but these things can get pretty
deep and they do some pretty amazing stuff.
The question that I always get from business people is,
when should I use it.
That's an important question because everyone's like,
"Hey, you should use AI, just use AI,"
and you're like, "I don't think you know what this is?"
We're making a function in different ways.
So the example I like to give it,
maybe this will be helpful to you when you talk to your manager
about what this can do is the example of a check.
Anyone who work for a bank?
Did I just make the thing go away, the screen.
You still go to my screen?
Okay, good. So anyone who work for a bank?
There's always a couple of you. The people who work for banks are
like programmer saints because the data they deal with,
it should be on the Dirty Jobs show.
It's like they're using COBOL.
For some reason, Bill always puts the name
and the age field and COBOL lets you do that,
and that's what they have to deal with.
So imagine you work for a bank and you show
up one day and the guy comes down,
and he's like, "We have a really good idea.
We want you to write an algorithm that detects whether
the pictures of the check from the users are upside down or not."
That's an easy problem because your bank is cheap and
all of your banks checks have
a black stripe at the top. Can you solve that?
You can, and you-all know that you just sample
100 random pixels at the top and if they're all black,
good. If they're not, you switch.
So you tell Steve for marketing, "I can do this,
can take six hours."
It really can take three,
but saw YouTube to watch.
Someone has to watch all that stuff.
Although Billy wants to talk about his life, we need to listen.
So you do this in one day and he comes back the next day,
it's ever a really even better idea,
and it's just this one thing.
Anytime a non-technical person tells
a technical person it's just this one thing, we have to sit down.
No one will go up to a civil engineer and be like,
"Hey, just this one thing,
turned to bridge upside-down."
But they will do that crap to us all the time.
So he goes just this one thing,
the owner of the bank, Sally,
loves her horses and she wants
a picture of her horse on the check, just this one thing.
Now, could you write an algorithm to
solve that check upside-down problem?
You could, you would ask for the check,
you would do the bunch of If statements and
you'd go home and take an extra long shower
because he wrote that code, but it works.
Now, imagine if you have to do this for any person,
for any check, for any picture.
Did you feel that gap as a programmer?
You felt it. That's the gap where machine learning works,
because programmers have this intuition about
programming and when you put that gap into their head,
they're like, "Oh, I can't write that."
No, you can, but you can give
the machine examples such that it can produce models for that.
That's how I explain that gap because intuitively,
programmers will start to code in their head,
black stripe at the top,
horse has pixels over here and
over here and I can fake that somehow,
and I can do some transformations,
but then when you say any picture,
any check, any person,
they can no longer do that,
but they know that a computer must be able to somehow.
This is where we say,
there's where machine learning works.
That's how I explain it.
Now, all of you know machine learning and so
I'm explaining it to you this way because you already know,
but when I talk about it to people that don't know,
I literally make programmers feel
the pain in that gap and that's where machine learning works.
Because as you know, programming is we make a function,
we give it some input, and out come answers,
in machine learning, we swap that around.
We don't call it a function, we call it model,
we don't call it answers and input,
we call it data in machine learning.
This you all know, it's just a different way to
think about it when you're explaining it to
the programmers who don't appreciate
the pain we go through everyday.
Like if something works at first time, we're different.
We know something's very wrong.
Programmers are like, "I'm a freaking genius."
We're like, "Oh, something's terribly wrong,
how is this possible?" What goes in there?
So again, what we're doing is we're making a random, not a random,
we're making a function such that we can
predict on the things that we're going to build,
but it's not clear to some people what these things look like.
To you it is and so I'm going to go
through an exercise to motivate from
first principles how it
works and we're going to use images like this.
Okay. They're very stupid.
This is stupid, but we're going to build all the way up
to convolutional neural networks from the bottom-up.
If you want me to go faster, remember.
So I'm going to ask you to build a machine learning algorithm,
and you wouldn't do this,
that predicts whether these pictures are
darker at the top or darker at the bottom.
Okay. How would you do that as a regular programmer?
By the way, this is what the data looks like.
So for example, this one is
darker at the top because there's the answer.
This is darker at the bottom and there's that perspective.
How do you know this was darker at the top?
You can use your regular programmer lizard brain.
I used to teach high school so
awkward silence is only awkward for you.
Everybody is like, "What?"
It makes me feel awkward.
Not me. Yes, what would you do?
So the answer that he gave was at the top,
at the bottom, and compare.
Easy. Some other people who,
because you all are smart,
you would come up with this,
other people take the average of
the top and the average of the bottom,
but as you know, mathematically,
it's a two-way divisions.
Easy. Now, what if I forced
you like to think of this intuition at the top, at the bottom,
what if I forced you to put it into something like this,
where I asked you to invent some magic W's
such that when you multiply it all of the input values,
you would get an answer that would say top or bottom?
What numbers would you put in for W's?
Yes.
111 here, zeros, minus 1.
That's super clever because
basically what it does is it adds the top,
adds the bottom, negates one side, and then they fight.
Because as I learned in fourth grade,
and this is magically served me very well,
there's no such thing as
subtraction, only addition of negative numbers,
saved my life as a third grader.
I remember getting all the minuses
and putting a plus and then a negative.
I'm like, "Okay. I know how to do this."
If you think about it, if it's more negative over here,
then it is positive over here,
the answer will be negative,
we guess bottom, otherwise we guess top.
Super easy. What if it's zero?
They're the same. Flip a coin. That's okay.
Any questions about this? See, this is where
I'm tricking you because
there's something hanging out over
there that you should have all seen.
This is supposed to be interactive. We're all friends here.
Canadians are now, "What's to b?" Great question.
Well, what if we wanted to guess
the bottom even though it's not darker at the top,
but it's within a couple of shades.
Like if it's dark enough then you add a negative number.
What that does is it forces this to be more negative,
but what if we want to do it to the top?
Then we add a positive number
and all of a sudden, we're guessing a top.
It's almost as if we're biasing our answer.
In machine learning, we're super clever.
We call that the bias.
Yeah, who knew? We're so clever.
This is simple and you've all seen this as a Perceptron algorithm.
It's the simplest thing,
but it's basically a dot product of the two things.
If it's negative, bottom,
positive, we guess top. Now, let's try it out.
Clearly works, you can see bottom right,
this number comes out to be 236, so we guess top,
this number comes out to be minus 234, we guess bottom.
Now, don't ever use machine learning for this problem.
That's dumb. We wouldn't do that.
So basically, and this is where everyone
else in the universe except for you all get surprised,
I say that W you just invented is a machine learning model.
I know, I'm trivializing our work, but it is.
If you think about all the other
machine learning models you build,
it's just that, but more of them.
What about top, middle, bottom?
What if I wanted to guess, well,
you just make three of them.
Now, instead of a single vector it becomes a matrix.
We do what I like to call
a cookie monster operator because I'm five years old.
You take this thing, you turn it on its side to go num,
dot-product three times gives you the answer.
The one with the highest number is the one that's the darkest.
This is why me and Dave were talking earlier.
So I hate when people say the output of models like this
have a probability like because
we soft-maxed over the end and we're like,
"Oh, it's not a probability."
We're forcing everything to sum to one mathematically.
So we're just basically whatever line
this dot product sends
the most density to whatever line of poo this is,
the one that has the most in it
is the one that's going to be closer to one,
but it sure as heck doesn't mean that this is 100 percent of cap.
So that's another thing that I like to put out there.
Top, middle, bottom, it's now a matrix.
What if we want to do digits?
Can we do this with digits?
Yeah, usually when something like N digit
get picture, we totally can.
The numbers are much bigger,
but it's the same thing.
Do you know if I run this on MNIST,
I get 92 percent accuracy?
What? I'm using an atom optimizer though,
and I'm being clever about how I'm doing it,
but if you run this stuff in TensorFlow,
you will get 92 percent accuracy on MNIST.
Yeah, just this.
It's the same thing that you were able to invent in
your head, which is pretty cool.
So how do we get all these W's and B's though?
[MUSIC]