Hello, and welcome to more More Data Mining with Weka.

This is Class 2.

We're going to look at two topics in this class.

In the first three lessons, we're going to look at Discretization, and in the last three

lessons, we're going to look at document classification.

Let's get going.

Discretizing is transforming numeric attributes to nominal.

There are a couple of obvious ways of doing this.

We've got a numeric attribute with a certain range.

We could take that range and chop it into a certain number of equal parts, or bins.

Just divide it into equal bins, and wherever a numeric value falls, we take that bin and

use its identification as the discretized version of the numeric value.

Or instead of using equal-sized bins, we can adjust the size to make the number of instances

that fall into each bin approximately the same: equal-frequency binning.

We're going to talk about those two things.

We'll talk briefly about the choice of the number of bins.

Then we'll talk about how to exploit the ordering information that's implicit in a numeric value,

but not implicit in a nominal value that you convert it to.

Let's look at equal-width binning.

I'm going to take ionosphere.arff, which has got a lot of numeric attributes.

I'm going to use J48.

I've set Weka up here with ionosphere.

I've run J48, and I get 91.5% accuracy.

Let's go and look at some of these numeric attributes.

The first one, a1, has got just two distinct values, 0 and 1, actually.

You can see the two values here.

The third attribute has got a bunch of different values ranging between -1 and +1, and kind

of scrunched up towards the top end.

The fourth attribute also varies between -1 and +1.

It looks like it could almost be a normal distribution.

I'm going to go to a filter here, an unsupervised attribute filter called Discretize.

Amongst the parameters here is the number of bins, and I'm going to use 40 bins.

And equal frequency--we're going to use equal-width binning, not equal frequency, leave that at false.

I'm going to run it, and look at the result.

Here is the first attribute from 0 to 1, just two values.

Here's the one that was all scrunched up to the top end.

This is -1, this is 0, and this is +1.

Here is the one that looked kind of normal.

You can see it is sort of normal-ish except for a bunch of extra values down here at -1

and +1.

I can look at those in the Edit panel, actually.

If I undo the effect of that, and go and look in the Edit panel and sort by this attribute,

you can see all the -1's here, and then a bunch of numbers, and then up at the top you

can see a bunch of extra +1's in this column.

Now I've applied the filter again.

I'm going to classify it and see what J48 makes of that.

We get 87.7% accuracy, which is not very good.

I can go back and change the number of bins.

I'm going to go straight to two bins here.

I'm going to, first of all, undo the effect of this, and then apply the two-bin version.

You can see that--well, this was two bins to start off with--but you can see that this

attribute, there's only two possible values, and this attribute is discretized into two bins.

If I run J48 again, I get 90.9%, which is pretty good, actually.

Going back to the slide, you can see the results for different numbers of bins here.

The last one, 90.9% is about the same, not too much worse than the original undiscretized version.

What's more, the tree has only got 13 nodes.

It's a much smaller, much more economical tree than the one we had before and very little

loss in accuracy.

So that looks really good.

I'm going to move now to equal-frequency binning.

Let's go back here, and take the discretize filter and change it to equal frequency.

I'm going to go back to 40 bins here, and I'm going to run that.

First, I need to undo the discretization, and then I'm going to apply this filter.

Well, it can't do much with the first attribute; that was binary to start off with.

But here, you can see that this is where they were all scrunched up towards the top end.

This is -1, this is 0, and this is +1.

You can see that, where possible, it's chosen the size of the bins to equalize the frequency.

It can't do anything with this large bin at the top, or this one at the bottom, or this

one in the middle, because all of the instances have +1 and here they've got 0 and here they've

got -1.

But where it can, it has kind of equalized the frequency.

This is the one that used to look normal.

You can see there's some extra -1's, 0's, and +1's, and it's kind of equalized the frequency

by choosing appropriate bin widths.

I can go and classify.

J48 gives me 87%.

It's a bit disappointing, not very good at all.

I can try with different numbers of bins.

Let me change this to two bins.

I need to undo this one first.

Then apply.

It hasn't done much here--which was originally just two bins--but you can see that here we've

got two equal-sized bins.

That's what histogram equalization, equal frequency, is trying to do: make bins with

the same number of instances in each.

If I just run J48 on that, I get 83%, which, again, is pretty disappointing.

Coming back to the slide, you can see that all of these equal frequency binning results

are worse than the original results.

The size of the tree is not hugely smaller, either.

So they're not really very good.

Which method should you use? How many bins should you use? Well, these are experimental

questions.

There's a theoretical result called "proportional k-interval discretization" which says that

the number of bins should be proportional to the square root of the number of instances.

That doesn't really help you very much in choosing the number of bins, because it doesn't

tell you what the constant of proportionality should be.

It's an experimental question.

A more interesting question is how to exploit ordering information.

In the numeric version of the attribute--and this is it at the top, the attribute value--we've

got a value v here, and there's an ordering relationship between different values of this attribute.

However, when we discretize it here into five different bins, then there's no ordering information

between these bins.

Which is a problem, because we might have a test in a tree, "is x

After discretization, to get the equivalent test, we would need to ask "is y=a?", "is

y=b?", "is y=c?" and replicate the tree underneath each of these nodes.

That's clearly inefficient, and is likely to lead to bad results.

There's a little trick here.

Instead of discretizing into five different values a to e, we can discretize into four

different binary attributes, k-1 binary attributes.

The first attribute here says whether the value v is in this range, and the second attribute,

z2, says whether it's in this range, a or b.

The third, z3, says whether it's in this range, a, b, or c.

The fourth says whether it's in the first four ranges.

If in our tree we have a test "is x

true and z4 is false.

So an equivalent test on the binary attributes are "is z3=true?".

If we take that tree we have before, testing on "x

Then we have the same kind of economy of the tree underneath this without replicating different

subtrees.

That's very easy in Weka.

We just go to our filter, and we set makeBinary to True.

You'll be using that option--and finding out that it's really pretty good, actually--quite

a bit in the activity associated with this lesson.

It makes sense, you know.

It allows us to retain the ordering information that's implicit in the original numeric attribute.

Here's what we've done.

We've looked at equal-width binning and equal-frequency binning, also called histogram equalization.

We briefly considered how many bins.

It's an experimental question.

We've talked about exploiting ordering information.

In the next lesson, we'll talk about taking the class into account, supervised discretization.

There's a bit in the book on discretization.

You should do the activity, and I'll see you in the next lesson.

Bye for now!