Hello,
and welcome to Advanced Data Mining with Weka.
I'm Ian Witten, and I'm going to be giving some of the lessons in this course, which
is brought to you by the entire data mining team at the University of Waikato in New Zealand.
This is the third in a series of online courses on practical data mining and the use of the
Weka workbench, and this course is going to look at the use of some popular packages in
Weka, specialist packages for specialist jobs in data mining.
It follows on from two earlier courses, Data Mining with Weka and More Data Mining with
Weka, and, like those, we're going to pick up some basic principles along the way, as
well as learning how to use Weka.
Also, we're going to look at some specific application areas in this course to give some
example applications.
As you know, a weka is a bird found only on the islands of New Zealand, but, as far as
this course is concerned, it's a data mining workbench--the Waikato Environment for Knowledge
Analysis--that contains a bunch of machine learning algorithms for various data mining
tasks like classification, preprocessing, feature selection, clustering, association
rule mining--things like that.
In this course, we're looking at versions 3.7 and 3.8 of Weka, 3.7 onwards, whereas
the previous courses looked at Weka 3.6.
These new versions have got a cleaner core, plus a system where you can install packages
that put in new functionality.
Some of these packages--and there's a lot of them--some of them do things that were
already in the core previously.
They have been stripped out to make a cleaner Weka core.
And there are lots and lots of other packages; we'll be looking at some of them, and users
can actually create and distribute their own packages if they want to put extra things in Weka.
So, what are you going to learn in this course? Well, we're going to learn how to use packages,
which is very easy.
We're going to look at the time series forecasting package and how to do time series forecasting
with Weka.
We're going to look at data stream mining: incremental classifiers in Weka, and we're
looking at the MOA system for Massive Online Analysis.
There's a MOA package for Weka, which we'll install and look at, and then also the MOA
system itself, you'll install that--its got a very similar interface to Weka--and look
at some its facilities for stream-oriented data mining.
We're going to be looking at Weka's interface to the R data mining system.
You can use facilities in R, which is a pretty advanced data mining system from Weka, getting
all that extra functionality right inside your Weka.
We're going to look at distributed processing using the Apache SPARK system.
We're going to be looking at scripting Weka in Python.
There's a package which allows you to script Weka right from the Explorer.
You can write little Python scripts, and also you can install the Python Weka wrapper, where
you install a full version of Python with access to all of the things that Python gives
you access to plus Weka besides.
And we're going to look at some applications.
We're going to look at analyzing soil samples.
We're going to look at neuroimaging with functional MRI data.
We're going to look at classifying tweets and classifying images and signal peptide prediction.
The aim of this course is to equip you to use Weka on your own data, and most importantly,
to understand what it is that you're doing.
We're assuming that you know about data mining and that you're a reasonably advanced user
of Weka.
You need to be an experienced Weka user to do this course.
You could do the early courses, Data Mining with Weka and More Data Mining with Weka,
those would certainly prepare you adequately for this course, but you don't have to do
those.
Just as long as you're an advanced user of Weka.
And if you're not sure, you can look at some of the videos on the YouTube WekaMOOC channel,
the videos from those earlier courses.
So you need to be a Weka user.
This is the team.
These are the Weka people at Waikato, and you'll meet all of these people as we go through
the classes and lessons in this course.
They're all experienced Weka users:
they've lived with Weka for many years.
Ok.
The course is organized just like the other courses.
There are five classes.
Each class corresponds to about a week.
This class, the first class, is on time series forecasting, and then the other classes--data
stream mining, interfacing to R, distributed processing and Python scripting.
Within each class there are six lessons, each lesson is a brief YouTube video, 5-10 minutes,
like this one, followed by an activity.
These activities you actually get to use Weka to do things yourself, and this is where you
do your real learning.
You don't really learn from me talking to you, you learn from actually doing stuff yourself.
So the activities are very important in this course.
The last lesson of each class is a particular application, where we just show you how Weka
has been used in a real application.
In order to get a certificate, you need to do the mid-class assessment and the post-class
assessment, which are worth one third and two thirds of the credit.
If you do sufficiently well in both of those, then you'll get a signed certificate from
me certifying that you've completed the course.
The activities are not required for assessment, but we strongly recommend you to do them.
As I said before, this is where you're going to be doing your real learning.
Now, what you should do now is to download Weka 3.7 or 3.8.
Actually, I'm not sure if 3.8 is out yet, as I record this video.
The way Weka works is that odd numbered distributions--
that's 3.7--are development distributions, and then
eventually they're released as stable distributions as even-numbered versions.
So Weka 3.7 will be released as Weka 3.8 either before you see this or soon afterwards.
Anyway, in any case, download the latest version of 3.7 or 3.8, if it exists.
That will be fine for this course.
You download it from this URL for Windows, Mac, or Linux.
You need to know how to do this, and the first course, Data Mining with Weka, will have taught
you how to do that.
It includes datasets for the course.
You do have to use Weka 3.7 or 3.8.
You can't use Weka 3.6 for this course because it doesn't have the package system.
The new stuff in this new version that we're looking at in the core we have some additional
filters and some things have been stripped out, some little-used classifiers have been
stripped out and moved into packages, and also little-used clusterers and association rule learners.
Also in the core, there are some additional feature selection methods.
But the most important innovation in the new Weka versions is the package system.
You go to the GUI Chooser and on the Tools menu you choose the Package Manager.
Let me do that and show you what happens.
I'm in the Weka Chooser.
Here's the Package Manager, and it actually goes to the internet to get a list of packages.
And here they are.
So, you can see there's a package for Arabic stemming here.
There's a package for the CLOPE algorithm.
A package for evolutionary search.
There's a lot of packages.
Let me just scroll through these.
There's just a lot of stuff here.
I think there's 150, 154 packages at the moment.
A lot of things.
Let's go back to the slide.
So, 154 official packages.
This packages list is on the internet.
You need to be connected to the internet.
When I just did that with Weka, and looked at the list of packages, it got them from
the internet to get the most up-to-date version.
As well as those official packages, there are unofficial packages, which are
user-supplied packages, and there's a list of those at this URL.
This is the first class.
We've been looking at Lesson 1.1, the Introduction and Installing Weka.
Next up is time series, linear regression with lags.
Then we're going to look at the timeseriesForecasting package and in Lesson 1.4 we're going to look
at forecasts, and then we'll look at lag creation and overlay data and then finally, we'll look
at an application: analyzing infrared data from soil samples.
That's it for now.
What you should do now is to go and do the activity associated with this lesson.
It's a revision activity where you'll get to do some of the more interesting questions
from the final test, the end of class test for the previous MOOC.
It will just allow you to exercise or refresh your Weka facilities and remember how to do
some of these things.
So off you go and do that, and I'll see you in the next lesson.
But, before I go, let me just show you that this is where I am at the moment.
I'm sitting here in New Zealand.
This is the world as we see it.
New Zealand's at the top, in the middle, where the red arrow is, and you're probably down
at the bottom somewhere.
This is where Weka is from.
I've turned this map of New Zealand around so that north is at the top, which is probably
more familiar to you, and we're right there.
The home of Weka at the University of Waikato in the center of the North Island of New Zealand.
I'm really looking forward to giving this course, and looking forward to seeing you
in the next lesson.
Bye for now!