Hi! Welcome back for another five minutes in New Zealand
with Data Mining with Weka.
This is Lesson 1.3, and we're going to look at exploring datasets
in this lesson.
We looked at this data file in the last lesson. It's the
weather data
toy dataset, of course. It has fourteen days, or
instances, and each instance, each day, is described by
five attributes,
four to do with the weather, and
the last attribute,
which we called the class value,
the thing that we're trying to predict, whether or not to play this
unspecified game.
This is called a classification problem.
We're trying to predict the class value.
Let's open up Weka.
It's here on my desktop.
I'm going to go into the Explorer.
We always use the Explorer.
I'm going to open the file.
I put the datasets in My Documents folder, so I can see them here.
Just open
the Weka datasets and the nominal weather data.
There's the weather data in Weka.
As we saw last time,
you can see the size of the dataset, the number of instances—fourteen—
you can see the attributes,
you can click any of these attributes
and get the values for those attributes
up here in this panel.
You also get at the bottom a histogram of the attribute values
with respect to the different class values. The different class
values are
blue for yes, play and
red for no, don't play.
By default,
the last attribute in Weka is always the class value.
You can change this if you like. If you change it here you can decide to
predict a different one other than the last attribute.
That's the weather dataset, and we've already explored that.
As I said, it's a classification problem, sometimes called a supervised learning
problem. Supervised
because you get to know the
class values of the training instances.
We take as inputted data set as classified examples,
these examples are independent examples with a class value attached.
The idea is to produce automatically
some kind of model
that can classify new examples.
That's the classification problem.
Here is what the examples look like. This is an instance, with
the different attribute values
a fixed set of features,
and then we add to that
the class to get the classified example.
That's what we have to have in our training dataset.
These attributes or features can be discrete or continuous.
What we
looked at in the weather data were
discrete, or we call them nominal,
attribute values where they belong to a certain fixed set,
or they can be numeric
or continuous values.
Also, the class can be discrete or continuous. We're looking at a discrete class,
yes or no, in the case of the weather data. Another kind of machine
learning problem would involve
continuous classes, where you're trying to predict a number.
That's called a regression problem
in the trade.
I'm going to have a look at a similar
dataset to the weather dataset.
The numeric weather
dataset.
Let me just open that in Weka,
weather.numeric.arff.
Here it is. It's very similar,
almost identical in fact,
for 14 instances, 5 attributes, the same attributes.
Maybe I should just look at this dataset
in the edit panel.
You can see here that two of the attributes—temperature and humidity—
are numeric attributes, whereas previously they were nominal
attributes. So here there are numbers.
What we see when we look at the attributes values for outlook, just as
before, we have
sunny, overcast and rainy.
For temperature, though, we can't enumerate the values,
there are too many numbers to enumerate.
We have the minimum and maximum value, mean, and standard deviation.
That's what Weka gives you
for the numeric values.
I'm going to look at a different dataset.
I'm going to look at the glass dataset, which is a rather more extensive dataset.
It's a real world dataset,
not a terribly big one.
Let's open it.
Here we've got 214 instances
and 10 attributes.
Here are the 10 attributes, it's not clear what they are.
Let's look at the class,
by default the last
attribute shown.
There are seven values for the class, and the labels of these values give
you some indication of what this dataset is about.
We have headlamps, tableware, and containers.
Then we have building and vehicle windows,
both float and non-float.
You may not know this, but there are
different ways of making glass, and
the floating process is a way of making glass.
These are seven different kinds of glass.
What are the attribute values?
I don't know what you remember about physics,
and I guess it doesn't matter if you don't remember.
RI stands for the refractive index.
It's always a good idea to check for reasonableness when you're looking at
datasets. It's really important to
get down and dirty with your data.
Here we're looking at the values of the refractive index—a minimum of 1.511,
a maximum of 1.534.
It's good to think about whether these are
reasonable values for refractive index. If you go to the web and have a look around,
you'll find that these are
good values for
the refractive index.
Na.
If you did chemistry, you'll recognize Na as sodium.
Here, it looks like these are percentages,
the different percentages of sodium.
Magnesium, Mg,
and so on. We would expect Silicon (Si),
to make up the majority of glass. It varies between 69.81%
and 75.41%.
These are percentages of different elements in the glass.
We can confirm our guesses here by looking at the data file itself.
Let me just find the glass data.
It's in Weka datasets,
and it's glass.arff.
This is the ARFF
file format.
It starts with a bunch of comments about
the glass database. These lines beginning with percentage signs (%) are comments.
You can read about this. We don't have time to read it now.
You can see about the attributes and it does say that
the attributes are
refractive index, sodium, magnesium, and so on.
And the type of glass, just like I said, is about
windows, containers, and tableware, and so on.
We can get down to the end of the comments,
and here we have stuff for Weka. This is the ARFF format. The relation has a
name,
you'll see it printed in the interface when you look.
The attributes are defined, they are real valued attributes,
numeric attributes.
The type
attribute is nominal, and the different values of type are
enumerated here in quotes.
That defines the relation and the attributes. Then we have an
'@data' line, and following that in the ARFF format, are simply the instances,
one after the other, with the attribute values all on one line, ending with
class by default. This is the
class value for the first instance.
I think there are 214
instances here.
There's the last one.
That's the ARFF format. It is a very simple,
textual file format.
Now we've confirmed our guesses about these numbers being percentages
and different elements.
We can think about
this some more. It's important then, that these numbers are
reasonable. If they went negative, for example,
that would indicate some kind of corrupted value. You can't have a negative
percentage.
We're expected silicon to be the majority component;
we're expecting the refractive index to be in this kind of range. It's always a good
idea when you get a dataset to just
click around in the Weka interface
and make sure things look real. Rather small amounts
of aluminum in glass. I guess that's not surprising;
I don't know very much about glass myself.
We're just kind of checking for reasonableness here—
a very good thing to do.
That's it then.
In this lesson, we've looked at the classification problem.
We've looked at the nominal weather data and the numeric weather data.
We've talked about nominal versus numeric attributes,
and we've
talked about the ARFF file format.
We've looked at the glass.arff
dataset,
and I've talked about sanity checking of attributes, and the importance of
getting down and dirty with your data.
If you'd like some further background on this, you can read Section 11.1
of the text and read about Preparing the data and Loading the data
into the Explorer.
Whether or not you do that,
please go and look at the activity associated with this lesson.
We'll see you soon. Bye!