welcome back to the data science with
MATLAB video series in this video we'll
work on predictive modeling or machine
learning alright so we went back and
performed our analysis over all of the
years we've done our pre-processing and
we've got a good subset of data to
continue to work with now we want to do
a prediction of the damage costs based
on the storm events and the other
variables that we have in our table that
we've been working with so we'll need to
do a couple of steps first we'll explore
different types of ways to approach the
problem then talk about data preparation
how to train and test the model and then
sort of what to do with it in the next
step so in this case we have a lot of
ways you know just generally speaking
with predictive modeling there are many
many ways to approach it depending on
your goals and the situation your your
data situation so you know for example
you could do curve fitting
classification regression neural
networks deep learning all of these
kinds of things and what's nice about
MATLAB use doing these techniques in
MATLAB is that oftentimes we have apps
that'll help you really explore and try
a bunch of different models as you're
going through this these tasks and so in
this case trying to predict our damage
cost and so that's a numeric prediction
so I'll use regression and before then
we want to make sure that we're in we
get the data in the right orientation
for regression or for machine learning
so let's take a look at the data that we
have just as a reminder so we have our
location the state the time information
the type of event the storm duration and
the damage costs and so we want to be
able to for our prediction we need to
capture all of the information that
might be useful to predict and so we
want to predict based on time but not
necessarily in a time series way we just
want to represent the time information
so we can just split up the year month
day and hours and use that in the model
to see
those are influencing the predict or the
costs so for example you know a blizzard
and July likely is more costly and so
we'll use that information in the model
another thing that we want to do is to
remove the infrequent classes and so in
this case we'll use will remove anything
that has less than 200 events so
whenever we're building models like this
we also want to partition the data so
that we have a data set held out for
testing so we'll split up our data into
training and testing sets and 70/30
split so now I'm gonna train our model
and we use an app and so here I'm just
I opened the regression learner and
we'll select the data so I'm just going
to select the training data set that we
created and I want to select the
response so that's what I'm trying to
predict which is our damage total and
then I can select or deselect the data I
don't need so for example I want I don't
need the property and crop damage or
latitude longitude because I'm
representing that data in a different
way with regression we've got a bunch of
different models and so if you have some
indication of what model you might want
to use you can select that or you know
maybe you know that it's linear you can
select all all linear in this case I'll
just select all of them why not
depending on how many cores you have
available you can use parallel and
that'll take advantage train many as
many models as you can at once and then
you can take a great look at the
visualizations you know even while it's
training to explore you know see some of
the patterns perhaps and then there are
some nice visualizations once the models
have trained to see to validate the
models so for example you can look at
residuals and you know examine the
predictions versus the actual data and
so the visualizations also help with
this you know for example some of these
categories have a lot of outliers
like our hurricanes and you know it's
it's able to predict the or the lower
values much better than the higher
values and so these are the kinds of
things that we want to look at in the
app and you know take advantage of by
training all of these different models
so once we're happy with a model we can
actually export it so we can start to
use it for predictions later and not
only can we export the model but we can
generate the code and that way we can
repeat the same exact steps that we went
through but using a function instead of
you know going through using the app
every time you can very quickly use the
function after this and also learn how
to adapt the model yourself I exported
the model from the app and now I'm going
to use it in our prediction so when you
export the model from the app it's
actually inside of a struct and it
contains information about the model and
even a function to help you predict
using that model taking the same steps
and the same variables as before the app
actually held out data for testing for
training and validation we also did that
before training the model and so now we
want to test the model on our test data
that we you know had not passed into the
app and so now I'll predict based on the
test data and I can use a histogram of
the errors or how far away the model
predicted from the real data whenever we
deploy this model and we put it into our
app we're gonna be getting the
information real time so we'll type in
the location and the type of event but
we basically just want to make sure we
have the same steps followed and the
data is in the same organization or the
same types that we originally trained
the model and so it's you know it would
be very expensive to have a tornado
right now in Massachusetts thanks for
watching in the next video we'll look at
scaling up our calculations to Big Data