Okay Michael so I think with that little
bit of discussion, I feel like we're done.
>> Cool! Alright. Well it's been nice talking to you. I hope
the course went well and, oh you mean just for this lesson?
>> Yeah just for this lesson.
>> Alright.
>> So let's.
>> Well let's wrap up this lesson then.
>> Yeah let's see what have we learned?
So remind me Michael what have we learned?
>> Well we were talking about instance based learning.
>> That's true. That's the first thing we learned. You will notice
by the way, I never actually told you why it was called instance
based learning.
>> Charles, why is it called instance based learning?
>> I don't know but I am willing to guess that it has to do with a
fact that we look at the exact instances that
we have and we base our learning on that.
>> Alright and we brought it up in
by starting off thinking about eager and lazy learning.
>> Right. What is the difference, Michael?
>> I will tell you when I need to tell you.
>> [Laugh]
That's exactly right.
>> So lazy learning is about putting off the work
until it's actually needed. Eager is about, as soon as
the problem is posed, solve it. And then, you know,
if you're lucky, the answer will eventually come in handy.
>> Exactly right. Okay what else?
>> So as a concrete example of a lazy
learner we talked about k-nearest neighbor or k-NN.
>> kNN. And
this whole notion of nearest neighbor. Is in
fact one way of talking about similarity functions.
>> Right and the similarity functions play a really central role in all this.
>> Right. Similarity. We talk about it as
if they were distance functions. But distance is
just another way of talking about, a similarity.
So, this is actually keeping. K-nn was a specific,
algorithm we used, and we talked about various versions of it,
and, the nearest neighbor part really got us to think a
little bit about similarity and distance and, and what all that
means. A really important thing here is, I think, that similarity is
just another way of capturing domain knowledge. And K in
k-NN is another way of capturing domain knowledge. And that
if we saw through the quizes and some of our discussions,
that this is actually very, very important. That domain knowledge matters.
Now, we also talked about KNN in the context of both regression and
classification. I see what you did there, 'Knnowledge', it's got KNN in it.
>> >Okay, so yeah, classification and regression are different things. But,
KNN can handle both of them. And, at the end of the
day, that's all stuck in our notion of similarity, and our
notion of averaging. Which we kind of took as an overall term,
for a bunch of different things you might do, which some people find confusing.
>> [LAUGH].
>> But I think that other people
would really kind of understand what we mean.
>> I'm very tempted to take that out of
Wikipedia, but that would, that would just be rude.
>> It would be rude. Anything else? We learned one big thing.
>> We looked at how to compose different learning algorithms
together. For example in the context of locally weighted linear regression.
>> Mm hm.
>> We used this instance based idea
along with linear regression to get something
that was both locally smooth but globally bumpy.
>> Right. So, I'm going to just say
locally weighted regression. Where we can do any kind
of regression we might want to. I was going to
call that $X. So, stick in your favorite value.
>> Let's see, what else? Oh, oh, a
really big thing was Bellman's curse of dimensionality.
>> Yes.
>> And
the idea there was that the more features that you
include The more data you need to fill up that space.
>> Yep, It's exponential.
>> And in fact I even just I decided to go and play with this a little
bit so that example that you were doing
before where the y equals x1 squared plus x2
>> Mm-hm.
>> When I gave it well as we saw in the example we gave it like ten or 12
examples and it did really badly. So it continued to do somewhat badly until
I got to about 100,000 and then it was actually doing [LAUGH] really well.
>> Hmm.
>> But that seems like an awful lot of
examples for what is otherwise a very simple problem.
>> Right. Well if you think about it, the
amount of data you have to see to determine
the relative. Relevance of the two different dimensions is
quite a bit in that particular kind of function.
>> Hm.
>> Yeah. That's a lot
of space to cover. All possible real
values [LAUGH] across a potentially infinite space.
>> Yeah, I guess that's true.
>> Yeah, so the cursor dimensionality is real and we
just sort of can't get around it. Although As I mentioned
earlier we will see in the second part of the
course ways that people try to get around the curse of dimensionality.
>> Ahah!
>> Mm-hm. Okay.
>> Or at least
blunt it. Right?
>> Yeah or at least blunt it because you can't
actually get around the curse of dimensionality, you can only deal with it.
>> There is no free lunch.
>> There is no free lunch. In fact, that's a theorem.
>> [LAUGH]
>> Isn't that a theorem?
>> Yeah, I think so.
>> What's the theorem?
>> No free lunch. That any learning algorithm that you create
is going to have the property; that if you average over all possible instances,
it's not doing any different than random.
>> Right. And, and another way of thinking about
that, a practical way of thinking about that is;
if I don't know anything about the data that
I'm going to have to learn over. Then, it doesn't
really matter what I do because there's all possible
kind of data sets. However, if I have domain
knowledge, I can use that to choose the best
learning algorithm for the problems that I'm going to encounter.
>> So does that mean that, that all of machine learning really comes down to,
you have to already know what you need to solve
the problem to apply these. Techniques to solve the problem?
>> No, but you have to know a little bit
about your problem in order to decide what to do.
And in fact, you could make the argument that this
entire class is about exposing the students to a wide
range of techniques and giving them enough practice so that
they can do a pretty good job of telling, given
a problem. Would it be better to use this kind
of technique or this kind of technique? Is K-nn a sort of
better way of approaching it? Decision tree a better way
of approaching it? Um,It's a lot of what this class is
about. Is helping them to get enough domain knowledge or enough
knowledge anyways so that they can apply it to particular domains.
>> Cool, so alright, that seems like a plenty useful lesson.
>> Yes, it's a very hopeful note to end on, so let's end on that.
>> Alright, see you next time.
>> Alright, bye Michael.