Hello! In the last lesson, we looked at using a classifier in Weka, J48.
In this lesson, we're going to look at another of Weka's principle features:
filters.
One of the main messages of this course is that it's really important when you're
data mining to get close to your data,
and to think about preprocessing it, or filtering it in some way
before applying a classifier.
I'm going to start by using a filter to remove an attribute from the weather data.
Let me start up the Weka Explorer and open
the weather data.
That's the one.
I'm going to remove
the humidity attribute.
That's attribute number 3.
I can look at filters, just like we chose classifier using this Choose button
on the Classify panel,
we choose filters by using the Choose button here.
There are a lot of different filters.
Allfilter and MultiFilter are ways of combining filters. We have supervised and
unsupervised filters. Supervised filters are ones that use a class value for
their operation.
They aren't so common as unsupervised filters, which don't use the
class value. There are attribute filters and instance filters. We want to remove an attribute,
so we're looking for an attribute filter.
There are so many filters in Weka that you just have to learn to kind of look around
and find what you want.
I'm going to look for removing an attribute.
Here we go, Remove.
Now, as before
when we configured the J48 classifier, we clicked here.
I'm going to click here, and we can
configure the filter.
This is "A filter that removes a range of attributes from the dataset".
I can specify a range of attributes here.
I just want to remove one. I think it was attribute number 3 we were going to remove.
I can invert the selection and remove all the other attributes and leave 3,
but I'm just going to leave it like that. Click OK,
and watch humidity go
when we apply the filter. Nothing happens until you apply the filter.
I've just applied it,
and here we are, the humidity attribute has been removed.
Luckily, I can undo the effect of that and put it back by pressing the Undo button.
That's how to remove an attribute.
Actually, the bad news is there is a much easier way to remove an attribute.
You don't need to use a filter at all.
If you just want to remove an attribute,
you can select it here and click the Remove button at the bottom.
It does the same job.
Sorry about that.
But filters are really useful and can do much more
complex things than that.
Let's, for example, imagine removing,
not an attribute, but let's remove all instances where humidity has the value 'high'.
That is, attribute number 3 has this first value. That's going to remove seven
instances from the dataset. There are
fourteen instances altogether, so we're going to get
left with a reduced dataset of
seven [instances].
Let's look for a filter to do that. We want to
remove instances, so it's going to be an instance filter.
I just have to look down here and
see if there is anything suitable.
How about RemoveWithValues?
The RemoveWithValues filter.
I can click that to configure it,
and I can click More to see what it does. Here it says it
"Filters instances according to the value of an attribute",
which is exactly what we want.
We're going to set the attributeIndex; we want the third
attribute, humidity,
and the first value. We can remove a number of different values; we'll just remove
the first value.
Now we've configured that.
Nothing happens until we apply the filter.
Watch what happens when we apply it.
We still have the humidity attribute there, but we have zero
elements with high humidity. In fact, the dataset has been reduced to only
seven instances.
Recall that when you do anything here, you can save the results. So, we could save that
reduced dataset if we wanted, but
I don't want to do that now.
I'm going to undo this.
We removed the instances where humidity is high.
We have to think about,
when we're looking for filters, whether we want a supervised or an unsupervised filter,
whether we want an attribute filter or instance filter,
and then just kind of use your
common sense to look down the list of filters to see which one you want.
Sometimes when you filter data you get much better classification.
Here's a really simple example.
I'm going to open the glass dataset
that we saw before.
Here's the glass dataset. I'm going to use J48, which we did before.
It's a tree classifier.
I'm going to start that,
and I get an accuracy of 66.8%.
Let's remove Fe,
that is, Iron. Remove this attribute,
and we get a smaller dataset.
Go and run J48 again.
Now we get an accuracy of
67.3%. So, we've improved the accuracy a little bit
by removing that attribute.
Sometimes the effect is pretty dramatic. Actually, in this dataset, I'm going to remove
everything except the refractive index
and Magnesium (Mg). I'm going to remove all of these attributes.
Left with a much smaller dataset with two attributes.
Apply J48 again.
Now, I've got an even better result,
68.7% accuracy.
I can visualize that tree, of course, remember by right
clicking here and visualizing the tree,
and have a look and see what it means. It is much easier to visualize the trees
when they are smaller.
This is a good one to
look at and consider what the structure of this descision is.
That's it for now.
We've looked at: filters in Weka;
supervised versus unsupervised, attribute versus instance filters;
to find the right filter you need to look;
they can be very powerful and judiciously removing attributes can
both improve performance and increase comprehensibility.
If you'd like, for some background reading on this, go to the textbook and
have a look at Section 11.2 on Loading and filtering files.
Then, go and do the activity associated with this lesson.
Bye for now!